Summary
✨ AI‑Generated
A cloud data engineer is needed to build scalable data solutions across two major cloud platforms. You will create API-driven ingestion processes, transform data for analytics, develop batch and near-real-time pipelines, orchestrate workflows, and implement monitoring, alerting, recovery, source control, and continuous delivery.
Highlights
Opportunity to design and maintain scalable multi-cloud data solutions, build reliable ingestion and processing pipelines, automate workflows, and apply modern DevOps practices. The role covers batch and near-real-time processing and end-to-end pipeline orchestration.
Description
Cloud Data Engineer (GCP and AWS)
Our Challenge
We are seeking a Cloud Data Engineer to design, develop, and maintain scalable data solutions on AWS and Google Cloud Platform.
The successful candidate will build reliable data ingestion and processing pipelines, automate workflows, and support efficient deployment through modern DevOps practices.
Responsibilities:
· Develop cloud data solutions using AWS Athena, Amazon S3, AWS Glue Data Catalog, and Google BigQuery.
· Design and implement API-based data ingestion and integration processes.
· Consume REST APIs, extract data, and transform it for analytical use.
· Develop batch, near-real-time, and change data capture (CDC) ingestion patterns.
· Build and maintain ETL/ELT data pipelines.
· Use Apache Airflow for workflow scheduling and pipeline orchestration.
· Manage pipeline dependencies, monitoring, alerts, error handling, and recovery.
· Use GitHub for source control and collaborative development.
· Develop and maintain CI/CD pipelines and release automation.
· Support deployment processes and promote consistent DevOps practices across data engineering initiatives.
Skills:
· 6+ years experience Developing cloud data solutions using AWS Athena, Amazon S3, AWS Glue Data Catalog, and Google BigQuery.
· Design and implement API-based data ingestion and integration processes.
· Consume REST APIs, extract data, and transform it for analytical use.
· Develop batch, near-real-time, and change data capture (CDC) ingestion patterns.
· Build and maintain ETL/ELT data pipelines.
· Use Apache Airflow for workflow scheduling and pipeline orchestration.
· Manage pipeline dependencies, monitoring, alerts, error handling, and recovery.
· Use GitHub for source control and collaborative development.
· Develop and maintain CI/CD pipelines and release automation.
· Support deployment processes and promote consistent DevOps practices across data engineering initiatives.
