Data Engineering Lead

Lead the engineering and technical strategy for a scalable multitenant data integration platform leveraging Apache Airflow and GCP technologies Job Description Lead engineering delivery and technical strategy for a multitenant data integration platformBuild and maintain productiongrade ETLELT pipelines orchestrated with Apache Airflow Utilize container platforms such as GKE and Cloud RunApply strong software engineering skills in Python APIs configuration management testing and performance tuning Implement CICD pipelines release controls automated quality gates and InfrastructureasCode TerraformDesign for reliability including retries idempotency deadletter patterns and operational resilience Translate complex technical concepts to diverse audiences and align crossteam efforts Work with GCP BigQuery for largescale data warehousing and analyticsArchitect cloud solutions using GCP Cloud Architecture best practicesDesirable Experience with data lineagemetadata management tools multitenant architectures data governance and vendor system integrations Roles and Responsibilities Own and drive the technical roadmap aligning engineering with business priorities and operational goalsLead solution design for new integrations and platform enhancements ensuring maintainability and scalabilityEstablish and enforce engineering standards for coding testing documentation observability and securityContribute handson to critical ETLs framework components design spikes and performance optimizationsDesign and develop robust Airflow ETL pipelines with effective scheduling dependency management and error handlingDevelop onboarding patterns for new source and vendor systems ensuring reliability and traceabilityDrive data quality controls including validation reconciliation and exception handlingEnhance the multitenant architecture focusing on tenant isolation configuration metadatadriven pipelines and runtime scalabilityOptimize platform performance cost and reliability across environments with capacity planning and SLAsImprove automation for build deployment and release processes with CICD best practices and selfservice toolingImplement comprehensive data lineage and observability with logging metrics ing and dashboardsPartner with stakeholders including product owners architects vendors and governance teams to manage requirements and deliveryLead agile ceremonies remove blockers and maintain delivery momentumMentor engineers through reviews and coaching to foster continuous improvement and capability growthSkills Mandatory Skills : Apache Airflow, ETL Concepts, GCP BigQuery, GCP Cloud Architecture, Python