top of page

Intermediate Data Engineer at Deimos

Nairobi, Kenya

Job Type

Full Time

Workspace

On-Site

About the Role

We are looking for a skilled Intermediate Data Engineer to join our Engineering division, contributing to a team focused on data-driven solutions. In this role, you will apply and adhere to established best practices to ensure the successful delivery of high-quality data products. You will be an essential member of a high-performing, interdisciplinary team and will report to an Engineering Manager.

What you will be doing

Assisting in the maintenance & improvement of the data lake for a leading African e-commerce group.
Collaborating with the team on defining data requirements, implementing solutions, improving processes, and performing domain-specific data analysis.
Performing critical assessments of data systems and architectures to pinpoint areas for enhanced data usability, operational efficiency, and customer satisfaction.
Developing robust, maintainable, and efficient data engineering code in line with industry best practices.
Identifying and addressing performance bottlenecks, troubleshooting data-related issues, and enhancing the efficiency of data processing and storage systems.

Requirements

Minimum of three (3) years of experience in data engineering, focusing on big data technologies and cloud-based data warehousing.

  • Proficiency in managing data storage and optimization in AWS S3 for data warehousing, handling both large volumes of raw and processed data.

  • Solid experience in developing scalable and efficient data processing workflows using Apache Spark on AWS EMR.

  • Good understanding of data integration and orchestration using AWS services such as Managed Workflows for Apache Airflow (MWAA) and AWS Lambda.

  • Knowledgeable in data modelling and architecture principles applicable to large-scale data environments.

  • Proficient in programming with Python or similar languages commonly used in data processing.

  • Familiarity with using Git for version control in a team-oriented project environment.

  • Experience in an agile development setting, with an emphasis on the iterative and incremental development of data projects.

  • Understanding of automated release management for data applications, employing CI/CD pipelines for streamlined deployment and operations.

  • Experience in setting up and managing data catalogs with AWS Glue Catalog to effectively organize metadata.

  • Desirable: Some experience with using Terraform for resource provisioning and deployments on AWS.

Application Link.

bottom of page