Summary
✨ AI‑Generated
Build and maintain scalable data pipelines and ETL/ELT workflows using Python and SQL. You will work with structured and unstructured data, develop warehouses, lakes, and data models, optimize processing performance, operate cloud data platforms, and implement data quality, security, governance, and access controls.
Highlights
Build scalable data pipelines and modern data platforms while collaborating with analysts, scientists, engineers, and business stakeholders. The role covers the full data lifecycle from ingestion and transformation to quality, security, and governance.
Description
Key Responsibilities
Design, develop, and maintain scalable data pipelines and ETL/ELT workflows.Build and optimize data ingestion, transformation, and integration processes.Develop efficient and reusable data solutions using Python and SQL.Work with structured and unstructured data from multiple sources.Develop and maintain data warehouses, data lakes, and data models.Perform data cleansing, validation, transformation, and quality checks.Optimize SQL queries and data processing workflows for performance.Work with cloud-based data platforms and services.Collaborate with Data Analysts, Data Scientists, Software Engineers, and Business Analysts.Monitor data pipelines and troubleshoot data quality, performance, and availability issues.Implement data security, governance, and access-control practices.Develop technical documentation for data pipelines, processes, and architecture.Participate in Agile/Scrum ceremonies and contribute to project planning and delivery.Identify opportunities to improve data architecture, automation, reliability, and scalability.Required Skills
Strong proficiency in SQL.Hands-on experience with Python.Experience developing ETL/ELT pipelines.Knowledge of data warehousing and data modelling concepts.Experience with relational and/or NoSQL databases.Experience with Apache Spark / PySpark.Experience working with at least one major cloud platform: Azure, AWS, or GCP.Understanding of data lakes, data warehouses, and distributed data processing.Experience with Git and version-control practices.Strong analytical and problem-solving skills.Good communication and collaboration skills.Preferred Skills
DatabricksAzure Data Factory / AWS GlueAzure Synapse / SnowflakeApache AirflowKafkaPower BITerraformDockerKubernetesCI/CDData governance and data qualityExperience with REST APIs and data integrationKey Technologies
Programming: Python, SQL
Big Data: Spark, PySpark
Cloud: Azure / AWS / GCP
Data Platforms: Databricks, Snowflake, Synapse
ETL/ELT: Azure Data Factory, AWS Glue, Airflow
Databases: SQL Server, PostgreSQL, MySQL, MongoDB
DevOps: Git, CI/CD, Docker
Visualization: Power BI / Tableau
