Summary
✨ AI‑Generated
A senior Python engineer is sought to own the architecture of a rapidly scaling data platform that transforms large volumes of raw real-world footage into structured, quality-controlled datasets. You will design ingestion, storage, processing, metadata, and delivery systems capable of handling multiple terabytes of data per day and progressing toward petabyte-scale infrastructure. The role provides broad ownership and close collaboration with company leadership.
Highlights
Fully remote senior engineering role with substantial architectural ownership and direct collaboration with founders. The position offers the opportunity to design a large-scale data platform handling terabytes of daily data, influence technical strategy, and build reliable, cost-efficient processing and quality-control systems.
Description
Location: Remote (Croatia) Type: Full-time
Our company helps turn real human work into training data for robotics.
We capture first-person footage of skilled workers and transform it into structured, quality-checked datasets that help robots learn to operate in the physical world.
We're hiring an exceptional, hands-on Python engineer to own the architecture behind our growth: how large volumes of footage enter our platform, move between storage and compute, become usable training data and reach customers reliably.
Your mission is to build a platform capable of handling multiple terabytes of incoming footage per day and growing toward petabyte-scale storage, with rigorous quality controls and sustainable processing costs.
You'll work directly with the founders, take ownership of the existing platform and shape its architecture as we scale.
You will:
- Own the complete system architecture: connect capture devices and ingestion stations, cloud storage, processing workers, metadata services, quality checks and dataset delivery.
- Design reliable ingestion at scale: resumable uploads, integrity verification, deduplication, bandwidth management and recovery from interrupted transfers across many recording sites.
- Architect storage across Backblaze B2 and AWS: define landing storage, processing access, archival, retention and restoration workflows.
Make deliberate decisions about durability, retrieval times, transfer costs and storage economics.
- Build distributed Python and GPU processing: design queues, scheduling, parallel execution, checkpointing, retries and backpressure so the platform handles growing workloads and recovers cleanly from partial failures.
- Own the video and ML pipeline: integrate and improve face blurring, privacy checks, hand-pose estimation, capture-quality assessment and dataset packaging.
- Protect data integrity throughout processing: preserve frames, timestamps and recording boundaries; track source files, model versions and processing history; make outputs reproducible and auditable.
- Measure and improve performance: identify bottlenecks across networking, storage, decoding, CPU and GPU workloads.
Improve throughput and cost per processed footage-hour while meeting quality requirements.
- Build operational reliability: establish monitoring, actionable alerts, automated quality gates, deployment practices and recovery procedures that let a small team operate a large platform.
We're looking for someone with:
- Exceptional Python engineering skills, including concurrency, multiprocessing, profiling, memory management and debugging complex production systems.
- Proven architecture experience with substantial data volumes: systems they personally designed, shipped and operated, with concrete evidence of scale and reliability.
- Strong knowledge of AWS, object storage and distributed systems, including storage lifecycle design, queues, access controls, failure recovery and cloud cost optimisation.
- Experience designing workflows that remain correct through duplicate events, interrupted jobs, worker failures and retries.
- Practical experience with video processing, GPU workloads or production ML infrastructure.
- Strong database and metadata modelling skills, with an understanding of how to keep large media assets and their processing records consistent.
- The judgment to make sound architectural decisions, implement them personally and take responsibility for how the system behaves in production.
Experience with Backblaze B2, AWS S3 and archival storage, PyTorch, OpenCV, FFmpeg, Modal, robotics datasets or multi-camera recordings would be especially valuable.
Our environment includes Python, GPU inference, cloud object storage, a Go ingestion layer and a TypeScript control plane.
You'll own the decisions that connect these components into a reliable platform and guide its evolution.
You'll join at a stage where your engineering decisions directly shape the company's capacity, data quality, and economics.
The role offers substantial technical ownership, close collaboration with the founders and room to grow into broader technical leadership as the team expands.
The position is full-time and remote, open to candidates located in Croatia.
To apply, send your CV and a short note on the largest data systems you've designed and operated to hire@hire-a.dev
