Summary
✨ AI‑Generated
A lead data engineering role focused on migrating an analytics platform toward a cloud-agnostic, on-premises-capable architecture. You will build streaming and batch Spark pipelines on Kubernetes, develop data-generation components, oversee deployments and testing, investigate defects, improve a rapidly evolving solution, and mentor team members. At least five years of experience with Spark, Kafka, and Kubernetes is required.
Highlights
Lead-level ownership of a cloud-agnostic data platform migration, hands-on work with streaming and batch processing, modern containerized data infrastructure, technical mentoring, and broad involvement across development and deployment.
Description
We are looking for a Lead Data Software Engineer to drive the migration of an existing data analytics platform, originally built on Azure resources such as Data Factory, Databricks, Event Hub, and Cassandra, to a cloud-agnostic platform deployable on-premises.
Responsibilities
Implement streaming and batch Spark pipelines running on KubernetesLead the implementation of objects within the data generator frameworkOversee deployment and testing across local and development environmentsDrive improvement of the current, actively changing solutionConduct bug investigation and resolutionPerform unit testing to ensure solution qualityParticipate in refinement, planning, and demo sessionsGuide and mentor team members throughout the implementation phase
Requirements
5+ years of experience with Apache Spark, Confluent/Apache Kafka, and KubernetesProficiency in PythonFamiliarity with the data engineering domain and cloud-agnostic platform migrationsCapability to dive deeper into new technologies in a short time during active implementation phasesAbility to work independently with syncs with a team leadStrong communication skills and proactivenessShowcase of being a hands-on team player and technical leaderProficiency in English at a B2+ level
Nice to have
Knowledge of JavaFamiliarity with Kafka Connect, KSQL, and Kafka StreamsSkills in Ansible and Argo CDUnderstanding of TDD
We offer
We connect like-minded people:Delivering innovative solutions to industry leaders, making a global impactEnjoyable working environment, whether it is the vibrant office or the comfort of your own homeOpportunity to work abroad for up to two months per yearRelocation opportunities within our offices in 55+ countriesCorporate and social eventsWe invest in your growth:Leadership development, career advising, soft skills and well-being programsCertifications, including GCP, Azure and AWSUnlimited access to EPAM's internal learning databaseFree English classes with certified teachersWe cover it all:Monetary bonuses for engaging in the referral programMedical & family care packageSix trust days per year (sick leave without a medical certificate)Coverage of psychology sessions of your choiceDiscounts for fitness clubs and sports programsBenefits package (sports activities, a variety of stores and services)
EPAM is global leader in AI transformation engineering and integrated consulting, serving Forbes Global 2000 companies and ambitious startups.
With over thirty years of expertise in custom software, product and platform engineering, we empower our clients to become AI-Native enterprises, driving measurable value from innovation and digital investments.
Experience the freedom of remote work from anywhere in Kyrgyzstan, whether it's the comfort of your home or our modern office in Bishkek.
