Senior LLM Inference Engineer

Join our AI team at Prosus, the largest consumer internet company in Europe and one of the biggest tech investors in the world. You'll be working on the team that drives growth and innovation across the company, with your work directly impacting how millions of people shop online. Who We’re Looking For We're looking for a Senior MLOps Engineer whose core expertise is LLM serving at scale. You'll own the infrastructure that gets models into production and keeps them running efficiently — vLLM deployment, inference optimization (quantization, batching, KV cache), GPU cost management, and production-grade APIs with strict latency SLAs. Pipelines and CI/CD matter, but this role is defined by serving performance and infrastructure depth, not pipeline orchestration. What You’ll Do Model Serving & APIs: Deploy and optimize LLM serving infrastructure using vLLMApply inference optimizations: quantization, continuous batching, PagedAttention, KV cache management to maximize throughput and minimize latencyDesign and build production-grade async API services (FastAPI, etc.) with pre/post-processing, business logic, and strict latency SLAsContinuously optimize serving costs through model compression, batching strategies, and infrastructure tuningImplement A/B testing infrastructure and canary deployments for safe model rollouts ML Pipelines: Build ML pipelines for data ingestion, processing, model deployment, and evaluationOwn CI/CD for ML systems, including automated testing, model versioning, and deployment workflowsImplement monitoring for model performance, latency, throughput, and costs with budget alertingSet up experiment tracking and model registry systems (MLflow, Weights & Biases, or similar)Define and monitor SLIs/SLOs for production model serving Infrastructure & Orchestration: Manage Kubernetes and Slurm clusters for GPU workloads with multi-tenant resource allocationOptimize GPU utilization and implement cost controls across training and inference workloadsOwn CI/CD pipelines, model versioning, and deployment automation Enablement & Best Practices: Create templates and documentation to accelerate team productivityEstablish MLOps best practices and guide teams in their adoptionSupport model training experiments when needed Minimum Qualifications Hands-on production experience with vLLM (or equivalent open-source LLM serving framework) — not managed services. You've tuned inference at the infrastructure level: quantization, continuous batching, KV cache, GPU memory management.5+ years in MLOps, platform engineering, or infrastructure with a focus on ML/LLM workloadsProven experience with GPU cost optimization: tracking, budgeting, alerting, and resource efficiency at scaleStrong Python skills with experience building production APIs (FastAPI or similar)Hands-on experience with Kubernetes and Docker for GPU workloadsExperience with job orchestration systems (Slurm, Ray, Argo, Kubeflow, or similar)Solid understanding of monitoring and observability for production ML systemsNaturally curious with a track record of proactively identifying and implementing improvements Preferred Qualifications Deep knowledge of GPU architectures and their performance implications for inference optimizationExpertise in model compression techniques: quantization (INT8, INT4, FP8), pruning, distillation for production deploymentUnderstanding of security best practices for ML serving: authentication, authorization, rate limiting, model access controlsExperience managing multi-tenant GPU clusters with fair scheduling and resource isolationTerraform or Pulumi for GPU-optimized infrastructure provisioning at scaleExperience supporting distributed training infrastructure: multi-node job orchestration, checkpoint management, debugging training failuresContributions to open-source MLOps tools or serving frameworks What We Offer Critical infrastructure ownership for high-impact AI projects that are strategically vital to the company, with direct visibility to senior leadership including the CEOState-of-the-art GPU infrastructure: H200 fleet, vLLM serving stack, cutting-edge optimization toolsExpert ML team who have released top Hugging Face models, published at NeurIPS, and built production systems that will run on your infrastructureSignificant autonomy in designing MLOps solutions, choosing tools, and shaping infrastructure strategy for LLM servingModern tooling: Latest MLOps frameworks, coding assistants, best-in-class development environmentHybrid work model with our Amsterdam office - home to the AI House, bringing together 200+ AI professionals through events and collaborationsCompetitive compensation, top-spec MacBook Pro, and an environment genuinely built for professional growth and learning If you want to own the infrastructure that runs some of the most demanding LLM workloads in Europe, let's talk. Our Diversity & Inclusion Commitment We respect the dignity and human rights of individuals and communities wherever we operate in the world. Building an inclusive workplace where everyone feels welcome and can thrive is critical for us. We provide access to education, which helps everyone understand the important role they play and the positive impact they can have. For a deeper look at our journey and future plans, explore our latest Annual Report. Stay up to date with our latest news to see what makes Prosus stand out. Learn more at www.prosus.com.