Senior Platform Engineer - Self-Hosted AI Platform

At EG, we develop software for our customers so they can focus on their profession. Our industry-specific software is built by peers from the industry, and backed by the scale of EG for stability, innovation, and security. We are committed to advancing industries by tackling big challenges such as resource use, efficiency, and sustainability. We are a thriving global workforce of 3000+ employees, with an 850+ strong team based in Mangaluru, India. We have a people-first culture fostering innovation, collaboration, and continuous learning. Join us in creating software that works for people, not software that makes people work. As Part of the Department We are seeking a highly experienced Senior Platform Engineer – Self-Hosted AI Platform to own and evolve the runtime layer of EG's centralized AI platform across hybrid and multi-cloud environments. This role focuses on building and operating a self-hosted AI platform that provides consistent, secure, and scalable access to AI capabilities across AWS, Azure, and on-premises environments. You will design abstractions that unify access to LLM providers and self-hosted models, ensuring a seamless developer experience regardless of where workloads run. As a senior platform engineer, you will drive architecture decisions, establish engineering standards, and ensure operational excellence across the platform runtime. You will collaborate closely with Azure and AWS platform teams to deliver a consistent and portable AI platform experience. Your Role: Own, Build, and Drive Own the design, deployment, and operation of the self-hosted AI platform runtime. Build and maintain platform components, including LLM gateways and routing layers (e.g., LiteLLM or similar), model serving and inference infrastructure, API layer and service abstractions, vector storage and retrieval systems, and caching and performance optimisation layers. Design a unified interface for accessing AI capabilities across cloud providers and self-hosted environments. Develop and maintain containerised services and runtime environments using Docker and Kubernetes where appropriate. Establish Infrastructure-as-Code and GitOps practices using Terraform, Helm, or similar tooling. Implement observability, monitoring, and tracing across platform components. Ensure platform security, including secrets management, access control, and data protection. Drive performance optimisation, scalability, and cost efficiency of AI workloads. Enable hybrid and multi-cloud interoperability across AWS, Azure, and self-hosted environments. Partner with SDK and application teams to ensure smooth integration and developer adoption. Lead incident response, production support, and operational excellence for platform services. Mentor engineers and contribute to shared platform standards across teams. Your Profile: A Detail-Oriented Platform Engineer with Strong Operational Ownership You have a Bachelor's or Master's degree in Computer Science, Engineering, or a related discipline. You have strong experience building and operating distributed systems and platform services. You have hands-on experience with containerisation and modern runtime environments (Docker, Kubernetes, or similar). You have experience designing platform abstractions or internal developer platforms (IDPs). You are familiar with AI/LLM ecosystems, including model serving, routing, or inference workflows. You have a strong understanding of APIs, service design, and multi-tenant systems. You have experience with observability, monitoring, and production operations. You have a solid understanding of security and compliance in distributed environments. You have experience supporting production systems with strong operational ownership. What Success Looks Like Deliver a robust self-hosted AI runtime that integrates seamlessly with AWS and Azure control planes. Provide a consistent developer experience for AI capabilities across all environments. Enable efficient routing between cloud-hosted and self-hosted models. Establish scalable and maintainable platform patterns adopted across EG engineering teams. Ensure high reliability, performance, and security of platform services. Accelerate AI adoption across 150+ products by simplifying access to AI infrastructure. Contribute to a unified AI platform strategy that spans hybrid and multi-cloud environments. Our values and our promise to you Lead the Industry We bring deep industry expertise and help our customers lead their industry Make it Happen We take ownership, move fast and deliver real impact Stronger Together We collaborate across borders with high standards, trust, openness and no politics At EG, we build software that makes a difference. Our industry-specific software help people and businesses work smarter every day. From doctors spending more time with patients to builders running safer, more efficient projects. Our software is built by peers from the industry and backed by EG’s scale for stability, innovation, and security. If you’re ambitious, curious, and take pride in ownership, you’ll thrive here. You’ll be trusted to grow your skills, shape meaningful solutions, and make an impact together with talented colleagues across borders. Join us in creating software that works for people - not software that makes people work. Curious what it's really like to work at EG! Explore the life at EG, our values and meet some of your feature colleagues, working at EG Learn more about EG here.