Data Platform Engineer

FOX IT is a 100% Portuguese Consulting company, founded in 2018 with the aim of positively transforming the IT market in Portugal. We see our consultants as our greatest asset and we work to ensure they feel motivated, supported and valued on a daily basis. The return is obvious and our customers feel it, with a very low turnover compared to what is, unfortunately, “normal” in the market.

What you will be doing:

• We're looking for a Data Engineer to support and develop a data platform within a large\-scale international energy provider.


• Provide user support for the data platform, troubleshooting and fixing issues


• Support data pipelines and ETL/ELT processes to enable reliable and scalable data flows


• Collaborate with platform architects to ensure the data platform is efficient, secure, and highly available


• Improve data processes, monitoring, performance and platform efficiency


• Support data quality frameworks and monitoring to ensure accuracy, consistency, and completeness of data pipelines


• Support for data lakes, data warehouses, and data streaming solutions


• Work with stakeholders (data scientists, analysts, business users) to solve issues in the data platform


• Automate repeatable tasks and improve the data ingestion, transformation, and delivery processes


• Troubleshoot and optimize data workflows and SQL queries for performance and cost efficiency


• Document data platform changes, processes, and best practices


• Review, evaluate and approve data product releases to production

• Experience with data pipelines in a production environment


• Strong hands\-on knowledge of SQL and experience with distributed data processing (e.g., Apache Spark)


• Proficiency and knowledge of troubleshooting dbt, Apache Kafka, Apache Ignite, Apache NiFi and Apache AirFlow


• Proficiency in a programming language such as Python or Scala for data engineering tasks


• Experience with cloud data services (AWS)


• Familiarity with Grafana tools


• Knowledge of data governance, security, and compliance in a cloud environment


• Experience implementing CI/CD and testing frameworks for data workflows


• Strong problem\-solving skills, communication skills, and passion for collaboration


• Fluent in written and spoken Portuguese and English.

• Familiarity with Dremio data lake warehouse architectures and concepts


• Experience with Terraform provisioning data services


• Familiarity with Power BI on\-premises gateway


• Hands\-on experience with data catalogue and governance tools (e.g., Collibra, Alation)


• Experience with machine learning pipelines and feature engineering.

Este anúncio é de Indeed. Ver anúncio original ↗