DontApply.Club
Login Register

Senior Data / Platform Engineer

Softnice Inc — United Kingdom · Posted ~1 hour ago

Senior

Skills

Databricks Delta Lake Python PySpark Azure Synapse ADLS Gen2 Azure Key Vault Azure networking RBAC CI/CD GitHub Actions Azure DevOps Infrastructure as Code Pulumi Terraform Bicep PowerShell Bash Azure Monitor Log Analytics KQL Parquet ETL/ELT

🔓 Log in to save this job, tailor your resume & track your apply process — 7 days free, no card needed.

Log in to add to target list

Summary ✨ AI‑Generated

Join a senior engineering role focused on building and operating modern cloud data platforms. You will work extensively with Databricks and Azure, develop data pipelines with Python and PySpark, automate deployments through CI/CD and infrastructure as code, and establish strong security, monitoring, and observability practices.

Highlights

Strong focus on modern Azure data platforms, Databricks, cloud infrastructure, automation, observability, and infrastructure as code, with opportunities to work across both data engineering and platform engineering.

Description

Must Have Databricks: hands-on experience building and optimizing pipelines, managing Delta Lake, and administering workspaces (cluster policies, Unity Catalog, Secret Scopes, Workflows). Python / PySpark: strong programming skills for data processing, automation, and scripting. Azure data stack: Synapse, Databricks, ADLS Gen2, Key Vault — including Linked Services, Managed Identity, and Spark Pool configuration. Azure platform fundamentals: compute, storage, networking (VNets, NSGs, Private Endpoints), identity and RBAC. CI/CD: GitHub Actions (with OIDC federation) and/or Azure DevOps for data and platform deployments. Infrastructure-as-Code: Pulumi (or Terraform / Bicep) across multiple environments. Scripting: PowerShell and Bash for platform automation. Monitoring & observability: Azure Monitor, Log Analytics, KQL. Big data file formats: Parquet and Delta Lake. Cloud-native data modelling and ETL/ELT frameworks on Azure. Good to Have Azure certifications: AZ-104, AZ-400, DP-203. Azure Data Factory (ADF), Microsoft Purview, Microsoft Fabric. Data governance: lineage, cataloguing, sensitivity labels. Event-driven architectures: Kafka / Azure Event Hubs. Delta Live Tables, MLflow. IDMC Secure Agent, Power Automate flows. Security baselines: CIS / NIST. Observability tooling: OpenTelemetry, Datadog. AI tools and their application in data engineering. Knowledge of the UK Insurance Market. Agile / Scrum delivery experience.