Software Engineer, Ingestion & Streaming

Datavant is the data collaboration platform trusted for healthcare. Guided by our mission to make the world’s health data secure, accessible and actionable, we provide critical data solutions for organizations across the healthcare ecosystem - including providers, health plans, researchers, and life sciences companies. From fulfilling a single patient’s request for their medical records to powering the AI revolution in healthcare, Datavanters are building the future of how data is connected and used to improve health. By joining Datavant today, you’re stepping onto a driven and highly collaborative team that is passionate about creating transformative change in healthcare. Join our Data & ML Platform organization on the Ingestion & Streaming team. We own the movement layer of Datavant’s data platform: batch and streaming pipelines, change data capture, document intake, and the self-service frameworks product teams use to land new sources into Snowflake, our Iceberg-backed lakehouse, and Databricks. Most data moves into the platform; some moves back out. Our job is to make both safe, fast, observable, and boring. This role is for an engineer who thinks like a platform builder first. We are shifting from a service-oriented posture (“we’ll build the pipeline for you”) to a platform-oriented one (“here is the paved path to build it yourself, safely”), though the team still builds pipelines directly when the situation calls for it. You will be central to that shift, designing the frameworks, tooling, and guardrails that scale how Datavant onboards new sources, and rolling up your sleeves for hands-on ingestion work when there isn’t yet a paved path. AI fluency is a baseline expectation here. You should already be using Claude Code, Cursor, Copilot, or equivalent tools as a core part of your daily engineering workflow, have opinions about how they make a team faster, and know how to apply them responsibly when PHI and other sensitive data are in scope. What You Will Do Design, build, and operate the ingestion frameworks that pull data from operational databases, vendor APIs, document streams, and third-party feeds into Snowflake, Iceberg, and DatabricksOwn and evolve the ingestion stack (AWS DMS, MWAA / Airflow, Fivetran, and the homegrown tooling on top) and design new patterns for API sources that don’t fit a managed connectorBuild self-service tooling so product engineers can onboard new sources without becoming experts in our infrastructureWrite and review the Terraform behind our ingestion infrastructure: AWS networking, IAM, compute, and data servicesPartner with product, data, and analytics teams to pick the right ingestion pattern for each source (CDC, batch, API, streaming) and stand it up end-to-endLead production troubleshooting and incident response, and turn each incident into a durable platform fixRaise the bar on engineering quality, observability, cost discipline, and security in everything the team shipsMentor mid-career engineers and pull peers along through code review, pairing, and design feedback What You Will Bring To The Table 2+ years in data engineering, platform engineering, or data-focused software engineering2+ years of hands-on AWS with real strength in networking (VPC, subnets, routing, PrivateLink, security groups), IAM (roles, policies, permission boundaries), and the data services this role touches, plus the judgment to know when to reach for what1+ years writing production Terraform or equivalent IaC, with experience owning modules, reasoning about state and blast radius, and shipping infrastructure changes safely1+ years building self-service tooling, internal platforms, or paved-path frameworks consumed by other engineersStrong SQL skills and the ability to reason about how data physically lives in a warehouse or lakeProduction experience with Snowflake (or an equivalent cloud data warehouse) and a workflow orchestrator (Airflow / MWAA preferred)Hands-on experience with at least one ingestion approach: CDC tooling (e.g., DMS, Debezium), managed connectors (e.g., Fivetran, Airbyte), or rolling your own pipelines for API sourcesSolid CI/CD discipline in GitHub or equivalent: branching, code review, automated checks, repeatable deploymentAI-native working style: daily use of Claude Code, Cursor, Copilot, or equivalent, with views on how they make a team fasterWorking knowledge of Python is expected; mastery isn’t the barClear written and verbal communication, especially in async, remote settings Bonus Points For Direct production experience with Iceberg or another open table format, especially bridging Snowflake and DatabricksHands-on Databricks or SparkKubernetes experienceSnowflake certification(s)Azure experience (we’re primarily AWS, but our customers and acquisitions aren’t always)In-depth experience integrating data systems with managed identity platforms, particularly via SCIM (SailPoint a plus)Prior experience in healthcare or another highly regulated industry like FinancePrior DBA, SRE, or DRE work operating production data systems under pressure Join Our Team at Datavant Ireland - A Galway-Based R&D Technology Center Datavant Ireland is an innovative R&D technology center based in Galway, dedicated to solving complex technology challenges with a purpose. We are looking for talented technical professionals who are product-minded and eager to make an impact. If you are passionate about developing cutting-edge solutions and want to be part of a growing team, we invite you to explore the exciting career opportunities at Datavant. Careers at Datavant offer the opportunity to apply your technology and problem-solving skills toward a vision that every healthcare decision is powered by the right data, at the right time, in the right format. We're looking for collaborative problem-solvers, curious learners, and thoughtful builders who are excited to make an impact. We value diverse experiences, perspectives, and approaches to solving challenges, and we believe the best outcomes come from teams that learn from and support one another. We provide team members with a robust total compensation package that includes: Competitive, equitable salary. Comprehensive benefits.Pension contribution.Flexible, Hybrid work environment. We are committed to building a diverse team of Datavanters who are all responsible for stewarding a high-performance culture in which all Datavanters belong and thrive. We are proud to be an Equal Employment Opportunity employer. Datavant is committed to working with and providing reasonable accommodations to individuals with physical and mental disabilities. If you need an accommodation while seeking employment, please request it here, by selecting the ‘Interview Accommodation Request’ category. You will need your requisition ID when submitting your request, you can find instructions for locating it here. Requests for reasonable accommodations will be reviewed on a case-by-case basis. For more information about how we collect and use your data, please review our Privacy Policy.