Principal Software Engineer - DevOps / Site Reliability Engineering

Riot Games was established in 2006 by entrepreneurial gamers who believe that player-focused game development can result in great games. In 2009, Riot released its debut title League of Legends to critical and player acclaim. As the most played PC game in the world, over 100 million play every month. Players form the foundation of our community and it’s for them that we continue to evolve and improve the League of Legends experience. We’re looking for humble but ambitious, razor-sharp professionals who can teach us a thing or two. We promise to return the favor. Like us, you take play seriously; you’re passionate about games. We embrace those who see things differently, aren’t afraid to experiment, and who have a healthy disregard for constraints. That's where you come in. The AI Efficiency team at Riot Games builds the platforms, tools, and technical foundations that help Rioters safely and effectively use AI to accelerate how we work. As these systems become increasingly important to creative, product, and development workflows across Riot, we need dedicated engineering leadership to ensure these systems remain stable, scalable, secure, and dependable in production. As a Principal DevOps / Site Reliability Engineer on the AI Efficiency team, you will own and evolve the operational foundations that allow the AI Efficiency team’s tech platform and the tools deployed within it to run reliably at growing scale. You will establish the systems, standards, automation, and support practices required to move quickly without compromising availability, deployment safety, maintainability, or user trust. You will partner closely with software engineers, ML platform engineers, technical artists, data scientists, and Riot’s infrastructure and security teams to improve developer experience, production readiness, observability, incident response, capacity planning, and service resilience. You will also help evaluate and operationalize AI-native engineering workflows such as agent-assisted code review, automated bug triage, AI-driven performance and security analysis, and browser-based UI validation. This role ensures the broader platform and its services are safely operated, supported, and continuously improved in production. You’re right for this role if you enjoy making complex systems reliable, reducing operational toil, improving how engineers build and ship software, and anticipating how systems will fail before those failures affect users. You are comfortable taking ownership of production health, leading through incidents, building sustainable operational practices, and creating paved roads that help teams move quickly and safely. You are also energized by the opportunity to responsibly bring new AI-native automation patterns into real engineering workflows, thoughtfully applying emerging capabilities to reduce friction, improve reliability, and enhance how engineers interact with production systems without compromising safety or control. Responsibilities Own and continuously improve the reliability, availability, scalability, performance, and operational health of the Efficiency team’s (web) platform and the tools deployed within itDesign, build, and maintain the infrastructure, deployment systems, and operational foundations required to support a growing portfolio of production AI services and internal toolsImprove CI/CD pipelines, release engineering practices, environment management, and deployment automation so software can be shipped safely, quickly, and consistentlyEstablish production-readiness standards and ensure new utilities have appropriate monitoring, alerting, ownership, documentation, rollback strategies, and support plans before launchDefine and operationalize service health indicators, SLIs, SLOs, error budgets, and reliability metrics that guide engineering priorities and tradeoffs between reliability, velocity, cost, and complexityBuild comprehensive observability across applications, infrastructure, service dependencies, and user workflows using metrics, logs, traces, dashboards, synthetic monitoring, and actionable alertsEstablish sustainable incident-management and on-call practices, including escalation paths, runbooks, severity definitions, communication protocols, and clear service ownershipLead or contribute to the diagnosis and resolution of production incidents, coordinating across teams and driving blameless post-incident reviews and durable corrective actionsBuild automation that reduces operational toil, improves mean time to detect and recover, and eliminates recurring sources of failure or manual interventionImplement safe deployment patterns such as automated validation, progressive delivery, canary releases, feature flags, health checks, rollback mechanisms, and controlled environment promotionPerform capacity planning, load testing, performance analysis, and resource forecasting to ensure the Toolkit can support increasing adoption and usage across RiotDesign and validate resilience, backup, recovery, failover, and disaster-recovery strategies for critical services, data, configurations, and infrastructureIdentify single points of failure and systemic risks across applications, cloud infrastructure, networking, databases, queues, caches, third-party dependencies, and operational workflowsImprove developer experience by building self-service workflows, reusable infrastructure components, local development environments, test environments, deployment tooling, and clear operational documentationEstablish and maintain infrastructure-as-code, configuration-management, secrets-management, and environment-governance practices that make infrastructure changes safe, repeatable, and auditablePartner with engineers throughout the software development lifecycle to embed reliability, operability, security, and maintainability into system design rather than addressing them only after launchTroubleshoot complex production issues across web applications, APIs, distributed services, containerized workloads, cloud infrastructure, network boundaries, authentication systems, and external service dependenciesPartner with ML Platform Engineers to ensure model-serving and inference systems integrate cleanly with the team’s broader observability, deployment, incident-management, and reliability standardsCollaborate with Riot infrastructure, information security, IT, developer-platform, and compliance teams to ensure the team follows appropriate operational and security requirementsEvaluate and implement AI-assisted operational workflows such as automated anomaly investigation, log analysis, remediation recommendations, regression detection, and runbook automationDefine guardrails, approval requirements, auditability, and escalation paths for agentic or automated operational systems that can interact with production environmentsChampion operational excellence through technical leadership, mentoring, documentation, standards, architecture reviews, and tooling that raise the reliability bar across the team Required Qualifications Bachelor’s degree in Computer Science or a related field, or equivalent professional experience5+ years of experience in Site Reliability Engineering, DevOps, Infrastructure Engineering, Platform Engineering, Production Engineering, Developer Experience, or a similar role supporting production systemsStrong programming and automation skills in one or more languages such as Python, Go, JavaScript, or TypeScriptExperience designing, operating, and improving cloud-based production systems in AWS, GCP, Azure, or comparable environmentsExperience building and maintaining CI/CD pipelines, release systems, deployment automation, and environment-management workflowsStrong understanding of observability practices, including metrics, logging, distributed tracing, dashboards, synthetic monitoring, and alert designExperience participating in or leading incident response, on-call support, root-cause analysis, and post-incident improvement workExperience improving the reliability, availability, scalability, and performance of distributed systems, service-oriented architectures, APIs, or web platformsStrong understanding of containerized environments and orchestration technologies such as ECS, Docker, Kubernetes, or comparable systemsExperience with infrastructure-as-code and configuration-management tools such as Terraform, Pulumi, CloudFormation, or similar technologiesWorking knowledge of Linux systems, networking, DNS, load balancing, service discovery, authentication, secrets management, and cloud security fundamentalsAbility to identify systemic operational risks and drive durable improvements across systems owned by multiple engineers or teamsAbility to collaborate across organizational boundaries, influence technical direction, and communicate clearly during both planned work and high-pressure incidentsExperience providing technical leadership, mentoring engineers, and establishing engineering standards across a team or organization Desired Qualifications Experience supporting AI/ML platforms, inference services, model-serving systems, GPU-backed workloads, data pipelines, or other compute-intensive servicesExperience defining and using SLOs, error budgets, and reliability metrics to guide prioritization and engineering decisionsExperience designing or improving internal developer platforms, self-service infrastructure, paved roads, golden paths, or shared engineering servicesExperience building sustainable on-call rotations and operational support models for services used by multiple teamsExperience with progressive delivery, canary deployments, blue-green deployments, feature-flag systems, and automated rollback strategiesExperience with performance testing, capacity modeling, chaos engineering, fault injection, resilience testing, or failure-mode analysisExperience designing backup, disaster-recovery, business-continuity, and regional failover strategiesExperience operating databases, caches, message queues, object storage, service meshes, API gateways, and other common distributed-system componentsExperience improving security posture through access controls, secrets management, dependency management, vulnerability remediation, network segmentation, and infrastructure hardeningExperience balancing availability, latency, engineering velocity, infrastructure efficiency, and cost in systems operating at scaleFamiliarity with browser automation and end-to-end testing frameworks such as Playwright for validating critical user workflows and detecting production regressionsExperience evaluating or integrating AI-assisted tools for incident investigation, anomaly detection, operational diagnostics, code review, test generation, or automated remediationFamiliarity with the risks and operational controls required when AI agents interact with source control, CI/CD pipelines, cloud infrastructure, or production systemsExperience establishing governance, approval workflows, audit trails, and quality controls for automated operational systemsExperience working in environments where experimental tools must be transitioned into reliable, supported, and maintainable production services For this role, you'll find success through craft expertise, a collaborative spirit, and decision-making that prioritizes your fellow Rioters, who are the customers of your work. Being a dedicated fan of games is not necessary for this position! Our Perks Full relocation supportComprehensive health insurance for you, your spouse, and childrenOpen paid time offRetirement benefits with company matchingLife insurance, parental leave, plus short-term and long-term disabilityPlay Fund so you can deepen your knowledge of our players and community through gamesWe’ll double down on your donations of time and money to non-profits