Skills
Data engineering
Data pipeline development
ETL
Python
SQL
Apache PySpark
Azure Synapse
Azure Data Factory
Azure Functions
Data modeling
Data warehousing
Distributed data processing
MLOps
Data integration
Data transformation
Data migration
Agile development
DevOps
Microsoft SQL Server
ADLS Gen2
Azure DevOps
Git
Summary
✨ AI‑Generated
An experienced data engineering opportunity focused on designing, developing, and optimizing modern data platforms and large-scale processing solutions. You will build complex pipelines and ETL processes, develop data models and warehouses, work with distributed processing systems, and support integration and migration initiatives. The role combines cloud data engineering with MLOps, DevOps, and modern architecture practices.
Highlights
Build and optimize modern, large-scale data platforms using a broad Azure and data engineering stack. The role offers senior-level ownership of complex pipelines, data platforms, distributed processing, integration and migration initiatives, while working with modern MLOps, DevOps, Agile, and cloud technologies.
Description
We are looking for an experienced Senior Data Engineer to join our team and contribute to the design, development, and optimization of modern data platforms and large-scale data solutions.
Key Responsibilities
Design, develop, and maintain complex data pipelines and ETL processes.Build scalable data solutions using Python, SQL, Apache PySpark, Azure Synapse, ADF, and Azure Functions.Develop and optimize data models, data warehouses, and large-scale data processing solutions.Work with SQL and NoSQL data stores, including MS SQL and ADLS Gen2.Develop and manage distributed data processing systems and clusters for batch processing.Implement data engineering solutions aligned with MLOps and modern data architecture practices.Support data integration, transformation, and migration initiatives.Follow Agile development practices and collaborate with cross-functional teams.Use modern DevOps tools such as Azure DevOps and Git to support development and deployment.Contribute to data platform performance, scalability, and reliability improvements.About You
8+ years of experience in software engineering/data engineering.Strong hands-on experience with Python and SQL; Java experience is desirable.Extensive experience with Apache PySpark, Azure Synapse, Azure Data Factory (ADF), and Azure Functions.Strong experience building complex ETL/data pipelines and working with modern data modelling concepts.Experience with MS SQL, ADLS Gen2, NoSQL, and large-scale data warehousing.Experience working with distributed systems and batch-processing environments.Knowledge of MLOps architecture and practices.Good understanding of Azure DevOps and Git.Strong understanding of ETL and data warehousing methodologies.Informatica experience is desirable.Strong problem-solving and communication skills with experience working in Agile environments.Apply Now
If you have strong Azure data engineering experience and are looking for an opportunity to work on large-scale data platforms and modern data solutions, we'd love to hear from you.
Location: Sydney
If this sounds like you, apply now or contact Anupama Cavuturi on 0486 199 174 for a confidential discussion.
