AZURE DATABRICKS DATA ENGINEER (HYBRID)

AZURE DATABRICKS DATA ENGINEER (HYBRID LISBON)

Portuguese company hires for hybrid position

Location: Lisbon, Portugal

• ️ Only candidates already based in Portugal will be considered

️ Language Requirements: English B2 and Portuguese B2

Seniority: Mid\-level (3\+ years)

Rate Between €2700 \- 3000 RV

• ️ Instructions: Please send your CV in English and make sure to include all skills and experience that match the requirements of the opportunity. This will significantly increase your chances of success

About the Opportunity

You will design, build and maintain enterprise data pipelines using Azure Databricks, dbt and Airflow/Astronomer. The role covers data ingestion, transformation, modelling, orchestration, quality, deployment and production troubleshooting.

Main Responsibilities

• Develop data pipelines using Azure Databricks, ADLS Gen2 and Delta Lake;


• Implement Medallion Architecture and transformation models with dbt;


• Create and maintain Airflow/Astronomer DAGs;


• Ingest data through APIs, SFTP/files and JDBC;


• Develop solutions using SQL, Python, Scala, Spark and PySpark;


• Implement data modelling, quality, metadata and lineage practices;


• Manage automated validations and deployments through Azure DevOps and CI/CD;


• Troubleshoot production issues and perform root\-cause analysis;


• Produce technical documentation and communicate with stakeholders.

Mandatory Requirements

• 3\+ years of relevant Data Engineering experience;


• Advanced SQL and strong Python skills;


• Production experience with Azure Databricks, ADLS Gen2, Delta Lake and Medallion Architecture;


• Strong dbt experience, including models, incremental materialisations, tests, dependencies and troubleshooting;


• Experience with Airflow/Astronomer, including DAGs, scheduling, retries, backfills and monitoring;


• Data ingestion experience with APIs, SFTP/files and JDBC;


• Knowledge of data modelling, SCD Type 2, surrogate keys, schema evolution, reprocessing and reconciliation;


• Experience with Spark/PySpark, Scala, Databricks Notebooks and Jobs;


• Experience with Git, Azure DevOps, Pull Requests and CI/CD;


• Knowledge of Data Quality, Metadata and Data Lineage;


• English and Portuguese at B2 level;


• Availability for hybrid work in Lisbon.

Nice to Have

• Unity Catalog;


• Microsoft Purview;


• Azure Monitor and Log Analytics;


• Microsoft Fabric;


• Power BI Semantic Models;


• Microsoft Entra ID;


• Azure Key Vault;


• Impala;


• Legacy Data Lakes and pipeline modernisation;


• Tableau;


• Pentaho.

Preferred Certifications

• Microsoft Certified: Azure Databricks Data Engineer Associate (DP\-750\);


• Databricks Certified Data Engineer Professional;


• dbt Analytics Engineering Certification;


• Astronomer Apache Airflow DAG Authoring Certification;


• Astronomer Apache Airflow Fundamentals Certification.

Ideal Candidate Profile

An autonomous Data Engineer with solid production experience in Azure Databricks, dbt and Airflow. The ideal candidate can independently design, deploy and troubleshoot enterprise pipelines while ensuring data quality, lineage and clear technical documentation.

Questions for Candidates

• How many years of Data Engineering experience do you have?


• What is your production experience with Azure Databricks, ADLS Gen2 and Delta Lake?


• Have you worked with dbt models, incremental materialisations and tests?


• Have you created Airflow/Astronomer DAGs with retries, backfills and monitoring?


• Do you have experience with APIs, SFTP/files and JDBC ingestion?


• Have you implemented SCD Type 2, data quality and lineage controls?


• What is your level of English and Portuguese?


• Are you available to work in Lisbon twice per week?


• What is your availability and expected rate?

CV Keywords

Azure Databricks, Microsoft Azure, ADLS Gen2, Delta Lake, Delta Tables, Medallion Architecture, dbt, Apache Airflow, Astronomer, DAGs, SQL, Python, Scala, Spark, PySpark, Databricks Notebooks, Databricks Jobs, APIs, SFTP, JDBC, Data Modelling, SCD Type 2, Surrogate Keys, Schema Evolution, Data Quality, Metadata, Data Lineage, Azure DevOps, Git, Pull Requests, CI/CD, Root\-Cause Analysis

\#00354124

Este anúncio é de Indeed. Ver anúncio original ↗