DontApply.Club
Login Register

Lead GCP Data Engineer

Frconsultancy — Australia · Posted ~2 hours ago

Lead Other

Skills

GCP Dataflow Cloud Spanner BigQuery Pub/Sub Cloud Composer Cloud Workflows Confluent Kafka Cloud Run Cloud Build Terraform Python Java data pipeline development data architecture GCS

🔓 Log in to save this job, tailor your resume & track your apply process — 7 days free, no card needed.

Log in to add to target list

Summary ✨ AI‑Generated

Lead the architecture and hands-on delivery of large-scale cloud-native data platforms. You will design and implement batch and event-driven pipelines, migrate existing platforms to managed cloud services, process real-time streams, and work across a broad GCP ecosystem with Python and Java. The role is available on a permanent or contract basis and requires full local work rights.

Highlights

Lead-level role focused on large-scale GCP data platforms, architecture, real-time and batch processing, and hands-on implementation using a broad modern cloud data stack.

Description

Role: Lead Data Engineer- GCP Type: Full time – Perm or Contract Location: Sydney/Melbourne Status: AU valid visa with full work rights Must have: GCP Dataflow, Cloud Spanner, BigQuery, Pub/Sub, Cloud Composer, Cloud Workflows, Confluent Kafka, Cloud Run, Cloud Build, TerraformProgramming knowledge and willingness to be hands-on - Python, Java Key Responsibilities In-depth knowledge and experience of GCP data and analytics technologiesDesigning Large scale Transaction Data Platform architecture for (a) migrating open source or public cloud data platforms to GCP cloud native services (b) designing greenfield data platforms on GCPDefine solution architecture and detailed design, and perform hands-on implementation for data pipelines for batch and event driven ingestion and processing of data from a variety of sources such as on-prem files, on-prem databases, APIs, etc.Real-time data ingestion and event-driven processing pipelines from Confluent Kafka and GCS to Spanner and BigQueryAutomatic transpilation of legacy code (HIVE, Teradata, python logic etc.) to BQ SQLBQ query performance optimisationCI/CD pipelines for data workloads using Cloud Build, Artifact Registry, TerraformOrchestration setup and Cloud Composer DAG development for data pipeline workflowsData governance solutioning using GCP governance tooling (Dataplex, Data Catalog) ○ Experience applying Generative AI technologies and integrations in enterprise environments.Tools and languages experience Required Interested? Please apply, thanks for your interest and time.