Senior Data Software Engineer

Wayve Technologies — United Kingdom · Posted ~8 hours ago

Senior Full-time Visa History ✓

Skills

Data Engineering Data Transfer Pipelines Distributed Systems Cloud Platforms Large-Scale Data Processing System Design Data Pipelines Cloud Infrastructure AI/ML Infrastructure

🔓 Log in to save this job, tailor your resume & track your apply process — 7 days free, no card needed.

Log in to add to target list

Summary ✨ AI‑Generated

Join a high-autonomy engineering team building mission-critical, globally distributed data infrastructure. As a Senior Data Software Engineer, you will design and operate massive-scale pipelines for video, LiDAR, and sensor data, make pragmatic architectural decisions, and help enable AI model development at unprecedented data volumes.

Highlights

Work on mission-critical, globally distributed data infrastructure at petabyte-per-day scale, with high autonomy and the opportunity to enable large-scale AI model training.

Description

The role Portal is the cloud platform that connects Wayve to the outside world: handling data exchange with partners, surfacing AI insights, and enabling the commercial deployment of Wayve's AI driver on vehicles across the globe. We're looking for a Senior Software Engineer, Data to help build Wayve's Data Transfer Hub: a mission-critical piece of infrastructure that will scale to transfer petabytes of sensor data per day from partners covering vehicles around the world. This is a pivotal project at Wayve. You'll work as part of a small, high-autonomy team building a globally distributed hub-and-spoke data exchange system, working with partners like Uber, Stellantis and Nissan. Your work will help make Wayve's AI model training possible at scale. Key Responsibilities Build and operate large-scale data transfer pipelines for video, LiDAR and sensor data from global partners at up to PB/day scaleContribute to the technical design of the Data Transfer Hub, making pragmatic trade-offs across UX, careliability, throughput, cost, partner constraints, observability and operational supportBuild parallelised, distributed data transfer pipelines using Flyte for workflow orchestration, Kafka/event-driven patterns for lifecycle tracing, and Azure for storage and transfer infrastructureHelp implement and evolve a globally distributed hub-and-spoke data transfer model to retrieve and share sensor data at scale with partnersWrite infrastructure-as-code in Terraform and pipeline logic primarily in PythonBuild reliable operational foundations, including retry strategies, backfills, data integrity checks, lifecycle/state tracking, dashboards and alertingWork on cloud-to-cloud and cross-cloud networking solutions, with Azure as the primary cloud platformOnce established, participate in an out-of-hours on-call and support rotation.Collaborate with product, commercial and partner-facing teams to understand constraints and deliver solutions that solve real customer and partner problemsWork in a fast-changing environment where requirements and partner deliverable dates evolve, while maintaining high engineering standards About you In order to set you up for success as a Senior Software Engineer at Wayve, we’re looking for the following skills and experience. Essential Proven experience building and operating large-scale data transfer, ingestion or data processing pipelines, ideally at multi-TB or PB scaleStrong software engineering skillsExperience delivering & maintaining production systemsExperience with workflow orchestration for parallelised jobs; Flyte preferred, but Airflow, Prefect or similar tools are also relevantExperience building reliable, observable data systems, including retry strategies, backfills, data integrity checks, lifecycle/state tracking and operational alertingStrong cloud platform skills, with Azure preferred; experience with AWS or GCP is a plusProficiency in Python and TerraformExperience with blob/object storage, large-scale file movement, networking or cloud-to-cloud data transferA product mindset: you think about the customer and build solutions that fix real problems, not just implement requirementsComfort working in a fast-changing environment with evolving requirementsA strong sense of ownership: you take responsibility for outcomes, communicate clearly, and help the team move quickly Desirable Experience working with external data partners or customer-facing technical integrationsBackground in autonomous vehicles, robotics, geospatial data, video processing or other data-intensive domainsExperience with Kafka or other event-driven systems for transfer observability and lifecycle trackingExperience building dashboards to communicate transfer job statusMulti-cloud experience across Azure, AWS and/or GCPExperience improving cost efficiency, throughput or reliability of large-scale cloud data systems This is a full-time role based in our office in London. At Wayve we want the best of all worlds so we operate a hybrid working policy that combines time together in our offices and workshops to fuel innovation, culture, relationships and learning, and time spent working from home.