Data Engineer

Accountabilities

• Design, develop, and maintain scalable and reliable data pipelines and Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes.


• Develop, maintain, and optimize data warehouse solutions and data models.


• Build and maintain data infrastructure primarily on Amazon Web Services (AWS), while supporting relevant legacy systems.


• Develop APIs and data integrations to enable effective data exchange across systems.


• Contribute to real-time data pipelines and processing using Apache Kafka.


• Automate and optimize data processes, workflows, and operational tasks.


• Monitor, troubleshoot, and continuously improve the reliability, performance, security, and data quality of pipelines and infrastructure.


• Work closely with analytics teams and other stakeholders to understand data requirements and deliver effective technical solutions.


• Maintain accurate and up-to-date documentation covering pipelines, infrastructure, workflows, and operational processes.


• Contribute to the modernization of legacy data processes, infrastructure, and overall data architecture.

• 2+ years of professional experience in data engineering.


• Strong SQL and Python programming skills.


• Hands-on experience with AWS services, particularly Amazon S3, Amazon Redshift, AWS Glue, and AWS Lambda.


• Practical experience with Apache Airflow for data workflow orchestration.


• Strong understanding of data warehousing principles and data modeling.


• Experience designing, developing, and maintaining production-grade data pipelines, with a strong focus on reliability, monitoring, and data quality.


• Solid understanding of both relational and non-relational databases.


• Familiarity with Git and Continuous Integration/Continuous Deployment (CI/CD) practices, preferably using GitLab.


• Experience developing APIs and building data integrations.


• Strong written and verbal communication skills.


• Upper-intermediate English proficiency or higher.


• Experience with Kafka and real-time data processing is a plus.


• Experience with Microsoft SQL Server and SQL Server Integration Services (SSIS) is a plus.

• Full-time position.


• Opportunity to work within an Analytics-focused environment.


• Remote working arrangement.


• Opportunity to contribute to modern cloud-based data infrastructure.


• Hands-on exposure to AWS, Airflow, Kafka, data warehousing, and data modernization initiatives.


• Opportunity to contribute to the improvement of data reliability, quality, performance, and architecture.


• Collaborative work with analytics teams and cross-functional stakeholders.

Deze vacature komt van Lever. Originele vacature bekijken ↗