Summary
✨ AI‑Generated
A large technology-driven organization is looking for a senior engineer to build scalable data platforms, streaming pipelines, and cloud-based analytics systems using modern data engineering tools.
Highlights
Lead development of large-scale data systems using modern cloud and big data technologies while applying advanced engineering practices.
Description
A leading retail corporation with a vast network of department stores is seeking innovative talent to enhance its digital offerings.
This company is a cornerstone of American retail, committed to delivering exceptional value and a seamless shopping experience through cutting-edge technology.
The Role
Lead the development of robust, observable, and measurable applications using extreme programming (XP) practices.
Develop, automate, and maintain batch and streaming ETL pipelines using Apache Airflow, Apache Spark, Python, and Scala.
Build and manage cloud-based data ecosystems on Google Cloud Platform (BigQuery, Bigtable, Dataproc, Pub/Sub, Cloud Storage, IAM, VPC).
Design and optimize SQL and NoSQL data models for data lakes and warehouses (BigQuery, MongoDB, Snowflake).
Apply modern Test-Driven Development (TDD) methodologies for big data pipelines, ensuring test automation.
Implement real-time ingestion with Kafka Connect and process streaming data using Spark Streaming or Apache Flink.
What You'll Need
4+ years of experience in software development.
Proficiency in Apache Airflow, Apache Spark, Python, and Scala for ETL pipeline development.
Experience with Google Cloud Platform services including BigQuery, Bigtable, Dataproc, and Pub/Sub.
Strong background in SQL and NoSQL data modeling and advanced query optimization.
Practical experience with Test-Driven Development (TDD) and continuous integration/deployment.
Understanding of application design patterns, event-driven architecture, and testing strategies.
What's On Offer
Opportunity to work on innovative solutions that elevate retail offerings.
Collaborative environment with design, product, and engineering experts.
Exposure to modern big data technologies and cloud platforms.
Chance to lead technical initiatives and shape product engineering standards.
Apply via Haystack today!
