DevOps Engineer - Big Data

Location: London, UK Working Mode: 3 days on site per week Description We're looking for a DevOps Engineer (Azure & Big Data) to join our team in London, UK You will play a key role in designing and implementing the global marketing data platform that consolidates client profiles and marketing preferences into a centralized master repository. This system will manage data pipelines from multiple sources, handle master data management elements, and distribute data via APIs and bulk feeds to marketing platforms. The role requires expertise in Infrastructure as Code (IaC), CI/CD automation, and experience in cloud-native and big data ecosystems. Responsibilities Design, implement, and maintain CI/CD pipelines leveraging tools like GitLab, Jenkins, or TeamCityEnable Infrastructure as Code (IaC) practices using Terraform for scalable deployments in AzureCollaborate closely with architects on end-to-end solution designs, ensuring alignment with enterprise architecture standardsManage Azure PaaS components, including Azure DevOps, Vault, and other security and configuration services (AKS experience is beneficial)Integrate and optimize data processing frameworks with Spark, Databricks, or Cloudera platform (Hue, Hive, HDFS)Support ETL pipelines built for master data management, customer profiles, and consent management workflowsEnsure environments are stable, secure, and optimized for high-volume data flows across multiple downstream systemsTroubleshoot environment, deployment, or automation issues quickly and effectivelyPartner with engineering and data teams to implement best practices for DevOps, governance, and cloud operations Requirements (Must Have Skills) Proven DevOps expertise in cloud environments, including IaC and CI/CD pipeline managementStrong hands-on skills in Terraform and scripting (Shell or PowerShell)Experience working with GitLab and integrating automation into build and release cyclesExposure to Azure Cloud and PaaS services, including Azure DevOps and VaultFamiliarity with data engineering and big data ecosystems, including Spark and Cloudera stacks (hue, hive, HDFS)Excellent problem-solving skills and ability to communicate technical solutions effectivelyPrevious experience working on data-driven projects and master data management solutions Nice to Have Skills Hands-on experience with Azure Data Factory and Databricks for advanced data processingKnowledge of AKS (Azure Kubernetes Services) for containerized workloadsBackground in development using Java, C#, or JavaScript for automation or service integrationKnowledge of customer data platforms, personalization strategies, and data governance requirements