Senior Data Engineer - Real-Time Streaming

You will be a senior engineer on the team that owns the real-time data platform. The platform turns operational events (orders, driver locations, geofence transitions, shifts, MQTT session events) into live analytics, routing inputs, and warehoused data across Postgres/TimescaleDB, ClickHouse, and BigQuery. You will own the platform end-to-end: design, implementation, deployment, and production operation of the streaming jobs, CDC pipelines, Kafka Connect bridges, and downstream sinks that move this data. Responsibilities Designing, implementing, and operating stateful Apache Flink streaming pipelines: keyed state, windowing, watermarks, timers, side outputs, custom sources/sinks Designing, developing, deploying, and maintaining Change Data Capture (CDC) pipelines using Flink CDC or Debezium, moving operational database changes into the streaming platform with correct snapshot/incremental handling, schema evolution, and downstream idempotency Building and operating Kafka and Kafka Connect pipelines: topic and partition design, source/sink connectors in distributed mode, schema and converter management, and deadletter routing Owning the path from Kafka → Flink → Postgres / TimescaleDB / ClickHouse / BigQuery, including schema design, idempotency strategy, batch tuning, and observability Diagnosing and fixing production issues: checkpoint failures, backpressure, state growth, sink slowness, autoscaler oscillation, restart loops Hardening the platform: delivery guarantees, watermarks, DLQ handling, schema migrations, alerting coverage. * Code reviews and mentoring mid-level engineers Contributing to the deployment side: Helm charts, ArgoCD applications, GKE configuration, Grafana dashboards, Prometheus alert rules Influencing direction: state backend choices, schema migrations, when a pipeline needs to be split or rebuilt Requirements 5+ years of professional software/data engineering experience, with strong Java expertise Production experience with Apache Flink and stateful real-time streaming pipelines Strong Apache Kafka experience, including Kafka Connect, consumer groups, partitions, delivery guarantees, and schema management Hands-on CDC experience using Debezium or Flink CDC Experience designing and operating Kafka → Flink → database/data warehouse pipelines in production Strong understanding of PostgreSQL and experience with at least one analytical database such as ClickHouse or BigQuery Experience with Kubernetes and deploying/operating production data workloads Proven ability to troubleshoot and optimize production streaming systems—backpressure, checkpoint failures, state growth, latency, lag, and sink performance Strong understanding of data consistency, idempotency, schema evolution, and observability Ability to work independently and own a data platform component end-to-end, from design through production operation Will be a plus Experience with TimescaleDB, hypertables, and time-series data Experience with PostGIS and geospatial/spatial query optimization Deep ClickHouse performance tuning, including MergeTree and partitioning strategies Strong BigQuery optimization and data modeling experience Experience with Flink Kubernetes Operator and Flink autoscaler tuning Experience with ArgoCD, Helm, GKE, Prometheus, and Grafana Experience designing high-throughput, low-latency real-time systems at scale Experience with MQTT or IoT/event-driven systems Knowledge of Kafka Schema Registry, Avro/Protobuf, and advanced schema evolution Experience mentoring engineers and leading technical decisions around streaming architecture Experience with routing, logistics, delivery, mobility, or location-based platforms would be particularly relevant What We Offer 20 days of paid vacation, 5 sick days, and public holidays Flexible schedule with a high level of autonomy Opportunity to influence product and technical decisions Professional growth and learning opportunities Comfortable working environment with a supportive team