Summary
✨ AI‑Generated
A technology organization is seeking a senior data engineer and architect to lead cloud-based data platforms, build scalable pipelines, develop analytics solutions, and support AI-driven applications.
Highlights
Senior role with ownership of advanced cloud data architecture, AI-enabled solutions, governance, and large-scale data engineering initiatives.
Description
We’re looking for a **Senior Databricks Data Engineer / Solutions Architect** to own an **Azure Databricks Lakehouse** from data ingestion and engineering through governed data products, **Power BI**, and **AI/RAG** solutions.
Key Responsibilities
• Architect **Medallion Lakehouse** solutions using Azure Databricks, Delta Lake & Unity Catalog
• Build scalable ETL/ELT pipelines using **PySpark, Spark SQL & ADF**, including CDC and data validation
• Design **dimensional models, data marts & Power BI** solutions
• Integrate data from databases, **REST APIs, SharePoint & SFTP**
• Build **GenAI/RAG** solutions using Azure OpenAI, Azure AI Search, FastAPI & Streamlit
• Develop **Azure DevOps CI/CD** pipelines and support Terraform across environments
• Develop, schedule & monitor **Python jobs on Databricks**
• Tune **Azure SQL / SQL Server** for performance, security & reliability
• Implement data quality, lineage, governance & **RBAC**
• Lead migrations from **on-prem SQL Server, SSIS & SSRS**
Required Skills
🔹 **8+ years** in Data Engineering / BI with **3+ years of Azure Databricks**
🔹 Strong **Databricks, Spark SQL, T-SQL & query optimization**
🔹 **Python, PySpark, SQL/T-SQL, Delta Lake**
🔹 **ADF, ADLS Gen2, Azure SQL & Synapse**
🔹 Medallion Architecture & Dimensional Modeling
🔹 **Power BI – DAX, Power Query & RLS**
🔹 Azure DevOps CI/CD
🔹 Strong database administration, performance tuning & security experience
🔹 Excellent stakeholder communication
Nice to Have: Databricks Data Engineer Professional, DP-203, AZ-305, Microsoft Fabric/OneLake, Terraform, **RAG/Vector Search/Embeddings**, or public-sector/regulated-environment experience.
