Staff AI Engineer - Inference Systems

What You Can Expect You'll design, implement, and own the inference systems that serve Zoom's AI models at production scale -- across real-time communication, vision, and language workloads. You'll be hands-on with kernel-level optimisation, inference framework internals, and production serving infrastructure, working closely with research and platform teams to push the boundary on latency, throughput, and cost. About The Team You will join a dynamic AI Infrastructure team focused on enabling high-performance AI across Zoom's products and services. The team builds the core systems that support model training, deployment, and inference at scale, driving innovation in areas such as real-time communication, computer vision, and natural language understanding. Responsibilities Design and build high-performance inference serving systems for large-scale transformer and multimodal models (including 100B+ and MoE architectures)Implement and tune inference optimisations: speculative decoding, continuous batching, KV cache management, prefill/decode disaggregation, and quantisation (INT4/INT8/FP8)Contribute to and customise inference frameworks (vLLM, TensorRT-LLM, SGLang, or equivalent) for Zoom's production requirementsWrite and profile CUDA kernels and custom ops where framework-level optimisation is insufficientOwn end-to-end deployment: from model packaging and serving API design to latency SLO monitoring and incident responsePartner with research to translate model architecture changes into inference-efficient implementationsDrive technical design and set the bar for inference engineering practices across the team What We're Looking For A Bachelor's or Master's degree in Computer Science, Electrical Engineering, or a related technical field, or equivalent practical experience5+ years of software engineering experience, with significant time spent on inference systems or ML infrastructure at production depthHands-on experience with at least one major inference framework: vLLM, TensorRT-LLM, SGLang, or ONNX Runtime (serving, not just export)GPU programming experience: CUDA kernel development, memory optimisation, and profiling with Nsight or equivalent toolsProduction experience serving LLMs or large vision models -- you've owned latency SLOs, debugged throughput regressions, and shipped optimisations that moved the needleDepth in at least two of: speculative decoding, continuous batching, KV cache design, quantisation pipelines, prefill/decode disaggregationStrong systems instincts in Python and C++; ability to read and modify framework internals Preferred Advanced degree (Master's or PhD) in a relevant technical fieldExperience with MoE models or 100B+ parameter deploymentsFamiliarity with disaggregated serving architectures or multi-node inferenceBackground in compiler-level optimisation (XLA, Triton, or similar) Minimum Salary Range or On Target Earnings: $206,600.00 Maximum $451,800.00 In addition to the base salary and/or OTE listed Zoom has a Total Direct Compensation philosophy that takes into consideration; base salary, bonus and equity value. Note: Starting pay will be based on a number of factors and commensurate with qualifications & experience. We also have a location based compensation structure; there may be a different range for candidates in this and other locations At Zoom, we offer a window of at least 5 days for you to apply because we believe in giving you every opportunity. Below is the potential closing date, just in case you want to mark it on your calendar. We look forward to receiving your application! Anticipated Position Close Date 10/16/26 Ways of Working Our structured hybrid approach is centered around our offices and remote work environments. The work style of each role, Hybrid, Remote, or In-Person is indicated in the job description/posting. Benefits As part of our award-winning workplace culture and commitment to delivering happiness, our benefits program offers a variety of perks, benefits, and options to help employees maintain their physical, mental, emotional, and financial health; support work-life balance; and contribute to their community in meaningful ways. Click Learn for more information. About Us Zoomies help people stay connected so they can get more done together. We set out to build the best collaboration platform for the enterprise, and today help people communicate better with products like Zoom Contact Center, Zoom Phone, Zoom Events, Zoom Apps, Zoom Rooms, and Zoom Webinars. We’re problem-solvers, working at a fast pace to design solutions with our customers and users in mind. Find room to grow with opportunities to stretch your skills and advance your career in a collaborative, growth-focused environment. Our Commitment At Zoom, we believe great work happens when people feel supported and empowered. We’re committed to fair hiring practices that ensure every candidate is evaluated based on skills, experience, and potential. If you require an accommodation during the hiring process, let us know—we’re here to support you at every step. If you need assistance navigating the interview process due to a medical disability, please submit an Accommodations Request Form and someone from our team will reach out soon. This form is solely for applicants who require an accommodation due to a qualifying medical disability. Non-accommodation-related requests, such as application follow-ups or technical issues, will not be addressed. Our interviews are supported by BrightHire, a tool that helps us create a consistent and thoughtful interview experience and may include recordings. Please refer to our candidate privacy statement for more information of how we use your data.