Summary
✨ AI‑Generated
A senior data engineer is sought to design and operate reliable pipelines that transform curated data into performant datasets and data products. The role uses Python, distributed processing, API services, automated testing, CI/CD, observability, and schema management while collaborating with product and AI engineering teams.
Highlights
Senior data engineering role focused on reliable, scalable pipelines and data products. Offers collaboration with product, AI, and architecture teams, opportunities to apply AI-assisted engineering practices, and strong emphasis on quality, observability, automation, and cost efficiency.
Description
We're hiring on behalf of a Haystack partner!
The Role
• Design and operate data pipelines that power SaaS products and partner integrations, transforming curated data into reliable, up-to-date, and performant datasets.
• Build and maintain robust ETL/ELT pipelines using Python with FastAPI services and Apache Spark/Scala workloads, focusing on data quality, lineage, observability, and cost optimization.
• Collaborate closely with Product Management, AI Engineers, and Architects to design Data Products enabling new AI-native features and partner integrations.
• Utilize AI tools and agents to accelerate pipeline development, schema migrations, and analyses, building reusable data engineering skills and workflows.
• Apply strong engineering principles to data management, including automated testing, CI/CD, observability, schema management, and clear ownership of pipelines and services.
What You'll Need
• Several years of senior-level experience in Data Engineering with Python and a proven track record of operating production data pipelines in regulated or business-critical environments.
• A degree in Computer Science, Data Engineering, Mathematics, Physics, or a comparable technical field with strong academic performance.
• Excellent SQL skills and experience with modern data stacks (e.g., Airflow, dbt, Spark/PySpark, Kafka) and cloud data platforms, ideally AWS.
• Proficiency in Infrastructure-as-Code with Terraform and GitOps/ArgoCD, along with experience in version control, testing, and monitoring.
• Demonstrated effective use of AI in software and data engineering, ideally with agentic tools, and experience in designing AI pipelines and context engineering.
• Fluency in German and English, strong problem-solving skills, innovative mindset, high proactivity, and an entrepreneurial spirit aligned with healthcare purposes and patient safety.
What's On Offer
• Competitive salary
• Flexible hybrid work model with options for remote work.
• Opportunities for professional development through an in-house academy and external partnerships.
• Access to health and wellness programs, including healthy canteen options and a fully equipped fitness studio.
Apply via Haystack today!
