Staff Software Engineer - Curated Data

Accountabilities

• Design and build the control plane for the curated data lifecycle, including dependency-aware orchestration, backfills, restatements, retries, partial-failure handling, and recovery mechanisms.


• Own architectural decisions across datasets, determining when a problem is best addressed through a data model, service, or job.


• Design and maintain clear contracts between data ingestion and curation layers so datasets can be understood and managed end to end.


• Build reliable alerting, monitoring, and data-quality signals that identify genuine issues while minimising operational noise.


• Work across Go, Kotlin, Rust, Python, and SQL, selecting technologies based on the requirements of each problem.


• Develop software that orchestrates and manages thousands of interdependent data models and supports large-scale data operations.


• Design systems capable of propagating schema changes and data corrections without unnecessarily disrupting downstream consumers.


• Translate ambiguous product requirements into technical designs, execution sequences, and actionable work for the broader engineering team.


• Personally implement the most technically challenging components while enabling other engineers to own and deliver complementary work.


• Debug complex production issues independently and lead root-cause analysis through to durable fixes.


• Contribute to technical architecture, engineering practices, and system design decisions across the curated data domain.


• Collaborate effectively with a distributed engineering organisation through clear written communication and a strong async-working approach.


• Use AI-assisted development tools thoughtfully to increase engineering productivity while validating outputs and maintaining high standards for correctness and quality.

• Significant experience as a backend software engineer with deep expertise in data systems, or as a data engineer who has developed strong software engineering capabilities.


• Proven track record of building and operating production-grade services, not solely data pipelines.


• Experience designing, building, or materially extending orchestration and scheduling systems, with a strong understanding of their failure modes and scaling challenges.


• Demonstrated experience with schema evolution and data correctness in systems where downstream consumers depend on reliable and stable data.


• Production experience with stateful stream-processing technologies such as Flink, Kafka Streams, Spark Structured Streaming, RisingWave, Materialize, or Feldera.


• Strong SQL and data-modelling capabilities, particularly when working with large datasets.


• Solid computer science fundamentals and a strong understanding of distributed systems.


• Experience reasoning about query execution and the underlying behaviour of data-processing engines is highly valuable.


• Strong debugging and root-cause-analysis skills, with a track record of driving issues through to durable production fixes.


• Strong architectural and systems-thinking abilities, with the confidence to make decisions across complex technical domains.


• Ability to take ambiguous requirements, develop a clear design and implementation sequence, and turn large technical problems into manageable workstreams.


• Strong written communication skills and the ability to collaborate effectively within distributed engineering teams.


• Practical experience using AI development tools and an understanding of their limitations, failure modes, and appropriate use in production engineering workflows.


• Experience with transformation frameworks such as dbt or SQLMesh is a plus, particularly experience extending them after encountering their limitations.


• Familiarity with data-lake formats such as Parquet, Iceberg, or Delta Lake is advantageous.


• Experience working in an organisation where data itself is a core product is a plus.

• Competitive salary and equity package, positioned within the top 25% of companies in the sector.


• Employee equity plan with employee-friendly terms, including a heavily discounted strike price and a 10-year exercise window.


• 5 weeks of paid time off plus local public holidays, with flexibility to swap holidays where appropriate.


• Fully remote-first working environment within a distributed international team.


• Flexible working hours, allowing you to structure your working day around your productivity and priorities.


• Strong focus on asynchronous collaboration and a deliberate approach to reducing unnecessary meetings.


• Private medical, dental, and vision insurance.


• 16 weeks of fully paid parental leave for primary caregivers and 6 weeks for secondary caregivers.


• Two-week phased return to work at full pay following parental leave.


• Quarterly company or team offsites at international locations.


• Annual travel allowance for connecting and co-working with colleagues.


• Home-office setup allowance, with local coworking space costs covered if preferred.


• Opportunity to work alongside experienced engineers and specialists in a highly collaborative environment.


• Company merchandise and team perks.

Deze vacature komt van Lever. Originele vacature bekijken ↗