Senior Databricks Engineer

Senior Databricks Engineer (12-Month Contract) About the Role We are partnering with a leading enterprise organisation undertaking a significant data platform modernisation initiative and are seeking an experienced Senior Databricks Engineer to join a high-performing delivery team on an initial 12-month contract. This is a hands-on senior engineering position suited to someone who has successfully delivered large-scale Databricks implementations, led data platform migrations, and can take ownership of complex data engineering challenges in enterprise environments. You will play a key role in designing, building and optimising modern data solutions while contributing to the adoption of emerging AI-enabled capabilities across the platform. Working closely with architects, engineers and business stakeholders, you will help shape scalable, high-performance data solutions built on modern cloud and lakehouse technologies. Daily Duties of the Role Design, develop and deliver scalable data engineering solutions using Databricks, Apache Spark and PySpark across complex enterprise environments.Lead the migration of legacy data platforms, warehouses and workloads into modern Databricks Lakehouse architectures.Build and optimise high-volume data ingestion, transformation and processing pipelines using Python, SQL and Spark technologies.Assess existing data architectures and provide technical recommendations to support platform modernisation initiatives.Troubleshoot and resolve complex data, integration and performance issues while ensuring reliability, scalability and maintainability.Collaborate with technical and business stakeholders to translate requirements into production-ready solutions and provide technical leadership throughout delivery. Skills Required for the Role Extensive commercial experience with Databricks, including the design and implementation of enterprise-grade data platforms.Strong hands-on expertise with Apache Spark, PySpark, Python and SQL.Proven experience delivering migrations from legacy data platforms and data warehouses into Databricks environments.Strong understanding of Delta Lake, Lakehouse Architecture, data modelling, data ingestion frameworks and transformation pipelines.Experience working within cloud environments including Microsoft Azure, Azure Synapse Analytics, and/or AWS.Practical knowledge of CI/CD, DevOps practices, performance optimisation, automated deployment processes and modern software engineering methodologies. Highly Desirable Experience with Databricks Genie.Exposure to Databricks AI/ML capabilities.Experience with Azure AI Foundry or comparable enterprise AI platforms.Knowledge of Generative AI, Retrieval-Augmented Generation (RAG) and Agentic AI solutions.Experience with Vector Search, embeddings and AI-powered data experiences.Ability to integrate AI capabilities into enterprise data platforms and analytics environments.