DontApply.Club
Login Register

Lead Data Engineer - GCP & MLOps

N2Sglobal — Australia · Posted ~13 hours ago

Lead Full-time

Skills

Data Engineering GCP MLOps Python Data Pipelines Apache Beam Kubeflow

🔓 Log in to save this job, tailor your resume & track your apply process — 7 days free, no card needed.

Log in to add to target list

Summary ✨ AI‑Generated

A technology organization is seeking a Lead Data Engineer to design data platforms, automate machine learning operations, and build scalable cloud-based solutions.

Highlights

Lead-level role building advanced data platforms and machine learning workflows using modern cloud technologies.

Description

Key Skills & Achievements Designed and implemented a customized Data Pipeline Framework using GCP Dataflow and Apache Beam.Delivered multiple AI/ML solutions on Google Cloud Platform (GCP) across various business domains.Implemented DevOps and MLOps practices for end-to-end automation, monitoring, and governance of data pipelines and machine learning systems.Led MIG implementation for a Reconciliation Dashboard solution.Developed a Dataflow Flex Template Framework and domain-specific Python libraries to accelerate data engineering workflows.Built and automated Kubeflow Pipelines for streamlined ML model training and deployment.Applied DevOps best practices to ML systems, ensuring scalability, reliability, and continuous delivery.Automated continuous deployment of Django applications on GCP.Implemented and automated Cloud Run deployments and serverless application management.Configured Google Secret Manager and Cloud KMS with deterministic encryption for secure data processing.Developed automated CI/CD workflows using Google Cloud Build.Performed BigQuery data extraction, preprocessing, and automated job execution for large-scale analytics workloads.Managed model lifecycle and maintenance using GCP Model Registry.Developed microservices and REST APIs for scalable cloud-native applications.Built customized web applications for AI Platform resource provisioning, orchestration, and management.Implemented automated continuous deployment pipelines for Google Cloud Functions.Deployed and managed analytics dashboards using R Shiny and Python Django.Optimized application code, cloud resources, and deployment processes to improve performance, maintainability, and operational efficiency.