DontApply.Club
Login Register

Azure Databricks Data Engineer

Bullit Management Services Limited — Australia · Posted ~8 hours ago

Senior Full-time

Skills

Python SQL Apache PySpark Azure Synapse Azure Data Factory Azure Functions ETL data pipelines data modeling distributed systems data warehousing ADLS Gen2 MLOps Azure DevOps Git Informatica

🔓 Log in to save this job, tailor your resume & track your apply process — 7 days free, no card needed.

Log in to add to target list

Summary ✨ AI‑Generated

Build and optimize complex data pipelines using Python, SQL, PySpark, and Azure data services. You will work with distributed processing, ETL, data modeling, SQL and NoSQL storage, large-scale data warehousing, and MLOps practices in an Agile engineering environment.

Highlights

Senior-level data engineering opportunity focused on complex pipelines, distributed processing, modern data modeling, and large-scale warehousing. The role provides broad exposure to Azure data services, MLOps, DevOps, and both SQL and NoSQL storage technologies.

Description

• Minimum 8+ years in development/software engineering and has extensively knowledge of working in Agile organisations. • Strong Python and SQL skill and good to have Java experience. • Have hands-on experience building complex data pipelines, • Have extensive experience with Apache PySpark, Python, Azure Synapse, ADF, Azure Functions, data models and ETL pipelines. • Have experience in setting up and optimizing both SQL and NoSQL data stores (MS-SQL, ADLS Gen2), as well as familiarity with object storage services (e.g., ADLS Gen2, Big Data), • Experience in managing and further developing distributed systems and clusters for batch processing, • Knowledge of MLOps architecture and practices, • Experience with modern data modelling concepts and large-scale data warehousing architecture. • Basic understanding of modern DevOps tool set, e.g. Azure DevOps, Git. • Knowledge of ETL and data warehousing methods, good to have Informatica experience.