Summary
✨ AI‑Generated
A Senior Data Engineer is sought to design and optimize scalable data platforms, ETL/ELT pipelines, warehouses, and lakes that power analytics, AI, and business applications. The role requires strong software and database fundamentals, expertise in data modeling and orchestration, and the ability to build secure, reliable, and high-performance data solutions.
Highlights
Senior data engineering role focused on building scalable and reliable data platforms that support analytics, AI, and business applications. The position combines software engineering, database architecture, data modeling, pipeline automation, performance optimization, security, and data quality.
Description
About the Role
We are seeking a Senior Data Engineer to design, build, and optimize scalable data platforms, ETL/ELT pipelines, and data warehouses that power analytics, AI, and business applications.
The ideal candidate has strong software engineering and database fundamentals, cloud experience, and expertise in data modeling, orchestration, and automation, with a passion for delivering reliable, secure, and high-quality data solutions.
Key Responsibilities
1.
Design, develop, and maintain scalable ETL/ELT pipelines to ingest, process, and transform data from diverse sources into data warehouses and data lakes.
2.
Build and optimize data models, database schemas, and storage solutions to support high-performance data processing.
3.
Manage relational and NoSQL databases, ensuring performance, scalability, security, and data integrity.
4.
Develop SQL objects including stored procedures, functions, triggers, views, and optimized queries for efficient data processing.
5.
Design and maintain OLTP and OLAP data systems, including dimensional modelling and data warehouse architectures.
6.
Implement data validation, monitoring, and quality assurance processes to ensure data accuracy and reliability.
7.
Develop and integrate REST APIs for data ingestion, distribution, and retrieval.
8.
Automate workflows and orchestrate complex data pipelines using Apache Airflow and other ETL automation tools.
9.
Collaborate with Software Engineers, Product Managers, and business stakeholders to deliver scalable data solutions.
10.
Contribute to data governance, security, documentation, and engineering best practices.
11.
Participate in code reviews, CI/CD processes, and Git-based collaborative development workflows.
12.
Continuously evaluate and adopt modern technologies, cloud services, and AI-powered development practices.
Job Requirements
1.
Strong proficiency in Python and SQL.
Experience with PL/SQL and/or T-SQL is preferred.
2.
Well-versed in at least one programming language such as Python, JavaScript, or similar.
3.
Experience with relational databases such as PostgreSQL, MySQL, SQL Server, Oracle, or similar.
4.
Familiarity with NoSQL databases such as MongoDB or Elasticsearch.
5.
Experience with cloud platforms such as AWS, Azure, or Google Cloud Platform (GCP).
6.
Familiarity with cloud services including S3, Redshift, Glue, Athena, RDS, Lambda, BigQuery, or equivalent.
7.
Experience with OLTP and OLAP database architecture, best practices, dimensional modelling, and data warehouse design.
8.
Working knowledge of Docker, containers, Kubernetes, and cloud-native infrastructure.
9.
Experience with Apache Airflow for workflow orchestration.
10.
Familiarity with DBT (Data Build Tool).
11.
Strong understanding of programming fundamentals, object-oriented programming, data structures, algorithms, and software engineering principles.
12.
Strong computing foundation with excellent analytical and problem-solving skills.
13.
End-to-end understanding of software architecture and application layers, including Frontend, Backend, and REST APIs.
14.
Knowledge of Power BI, Tableau, or other BI/visualization tools is an advantage.
15.
Proficient working knowledge of Git and Git-based development workflows.
Preferable Skills
1.
Experience with Prompt Engineering and Retrieval-Augmented Generation (RAG).
2.
Familiarity with GenAI orchestration frameworks such as LangChain, LlamaIndex, or similar tools for building and integrating AI-powered applications.
3.
Understanding of Large Language Models (LLMs), embeddings, vector databases, AI APIs, and AI-assisted development workflows is an added advantage.
4.
Fundamentals of Artificial Intelligence (AI) and practical exposure to Generative AI technologies are a plus.
Education & Experience
1.
Bachelor’s degree in Computer Science, Information Technology, Software Engineering, Data Science, or a related discipline is preferred.
2.
5+ years of professional experience in Data Engineering, Software Engineering, Database Development, or a related technical role.
3.
Relevant internships, open-source contributions, and personal projects demonstrating practical expertise are valued.
4.
Professional certifications are a plus, including but not limited to: AWS Certified Data Engineer – Associate, Google Cloud Professional Data Engineer, Microsoft Azure Data Engineer Associate
