Summary
✨ AI‑Generated
A technology-focused organization is seeking a data engineering professional to design and maintain cloud-based data pipelines. The role involves ETL development, data migration, SQL optimization, scripting, automation, and ensuring reliable data processing across enterprise environments.
Highlights
Opportunity to build scalable cloud data solutions, work with modern analytics platforms, optimize data workflows, and contribute to large-scale data transformation initiatives.
Description
GCP Experience Must
Role Responsibilities
· Build and maintain scalable data extraction, transformation, and loading workflows using Informatica
· Manage tables, datasets, partitions, and clustering inside Google BigQuery for large-scale analytics.
· Write and tune advanced SQL scripts and queries to boost execution speed and lower compute costs.
· Use Unix shell scripting to automate file transfers, pre- and post-load routines, and batch job sequencing.
· Develop custom Python scripts to parse files, orchestrate data movement, and interact with cloud APIs.
· Schedule, monitor, and troubleshoot integrated workflows across Unix cron jobs and cloud schedulers.
· Move legacy on-premise data assets into cloud environments via hybrid ETL pipelines.
· Implement error-handling, logging, and data validation rules using SQL and Python checks.
· Diagnose bottlenecks in Informatica mappings, BigQuery queries, and shell scripts.
· Maintain technical mapping specs, system architecture diagrams, and partner with analysts or data
Key Skills & Experience
· 4–7 years of hands-on data engineering experience designing, developing, and supporting production data pipelines.
· Experience building ETL/ELT workflows using Informatica, including mapping development, workflow scheduling, monitoring, and troubleshooting.
· Proven capability working with Google BigQuery, including dataset management, table design, partitioning, clustering, and query performance optimisation.
· Advanced SQL skills, including complex joins, window functions, stored procedures, query tuning, and data validation routines.
· Practical experience with Unix shell scripting for file handling, batch orchestration, job sequencing, and automation of operational tasks.
(desired)
· Proficiency in Python for data processing, file parsing, API integration, automation, and development of reusable data engineering utilities.
· Experience supporting cloud migration or hybrid data integration initiatives, including movement of legacy on-premise data assets to cloud platforms.
(desired)
· Good understanding of data quality, reconciliation, error handling, logging, and operational monitoring practices across batch and scheduled workloads.
· Ability to diagnose and resolve technical issues across Informatica mappings, SQL queries, shell scripts, and cloud data pipelines.
· Experience maintaining technical documentation, mapping specifications, operational runbooks, and working closely with analysts, testers, and business stakeholders.
· Familiarity with Agile delivery practices, code version control, release processes, and collaborative delivery across onshore and offshore teams.
· Bachelor’s degree in Computer Science, Information Technology, Data Engineering, or a related discipline; relevant cloud, data, or Informatica certifications are desirable.
