Summary
✨ AI‑Generated
A technology organization is seeking a hands-on Data Engineer to scale an enterprise data platform for AI and generative AI use cases. The role focuses on implementing unified governance, fine-grained access controls, metadata structures, lineage, and secure AI pipelines, including governed model and agent workloads. This is a remote-first contract requiring the candidate to be based in the UK.
Highlights
Remote-first contract opportunity focused on practical delivery of modern data and AI governance capabilities. The role offers hands-on experience with enterprise-scale data platforms, governed GenAI workloads, secure pipelines, and access-control architecture. Outside IR35.
Description
I'm working with a client who is scaling their Databricks platform to support AI and GenAI use cases, and they need people who can make sure it's governed properly from the start.
This is about hands-on Databricks delivery, not just certifications.
This is a remote first contract but you must be based in the UK.
This is outside IR35.
What they're building
Unity Catalog as the single governance layer across data, ML models and AI assetsGoverned GenAI and agent workloads using Mosaic AI and Agent BricksSecure, auditable AI pipelines from ingestion to model serving, all within the Databricks platform
Databricks project experience we're looking for
Unity Catalog implementations or migrations from Hive metastore, including metastore design, catalog and schema structure, and workspace and environment separationFine-grained access control in Unity Catalog: row filters, column masks, and attribute-based access control (ABAC) with governed tagsUnity Catalog lineage and system tables used for auditing, usage monitoring and compliance reportingData classification and sensitive data tagging in Unity Catalog for PII and regulated datasetsMosaic AI Model Serving and AI Gateway, including access controls, rate limiting, guardrails and usage trackingModels in Unity Catalog via MLflow: registry, versioning, lineage and permissions, plus MLflow tracing and evaluation for GenAI applicationsAgent Bricks or the Mosaic AI Agent Framework in production, with governance over the data and tools agents can accessDatabricks Lakehouse Monitoring and data quality expectations in Lakeflow Spark Declarative Pipelines supporting AI reliabilityDelta Sharing or Databricks Clean Rooms for governed data collaborationDatabricks Asset Bundles and CI/CD with governance policies built into deployment
Ideal background
Hands-on Databricks engineering or architecture experience on Azure, AWS or GCPDelivery in regulated or risk-sensitive environments such as financial services, public sector or healthcareAble to explain what you built in Databricks, the governance decisions you made and why
Why this one
Work on live AI projects with real governance requirements, not proofs of conceptExposure to the latest Databricks capabilities in productionA client who treats governance as a core engineering discipline
If you've delivered Unity Catalog or AI governance work on Databricks, or know someone who has, apply now!
