AI Infrastructure & LLM Inference Engineer

The RoleAs an AI Infrastructure & LLM Inference Engineer, you will build and operate self-hosted AI platforms for clients who want to run their own inference. As capable open-weight models and accessible GPU hardware keep arriving, more organizations are choosing to keep inference local or hybrid rather than sending everything to a hosted API. You will be the engineer who makes that practical: secure, fast, observable, and supportable. The work spans the full stack beneath the application. You will validate and configure GPU environments, tune the operating system and NVIDIA software stack, select and operate inference runtimes, stand up gateways and identity, build observability, and write the runbooks that let client engineers run the platform confidently after handoff. Deployments range from fully on-premises and air-gapped to hybrid architectures that pair local inference with cloud capacity. This is production inference and platform operations, not foundation-model research. You will deploy and optimize open-weight language and multimodal models, establish performance baselines that hold up under real load, and keep systems reliable under real usage. You will work shoulder to shoulder with other applied AI engineers who build the retrieval, chat, research, and developer experiences on top of what you establish, so the quality of your work shows up directly in what users experience. We care about what you have built and what you have run in production operations, not necessarily how long you have been running it. Many strong candidates will have roughly three to five years in infrastructure, platform engineering, SRE, HPC, distributed systems, or production ML, with hands-on LLM serving, local inference, or GPU-backed AI work. Years of experience are guidelines. We always look at actual work experience as the key indicator of experience and fit. What You'll Do• Design and Deploy AI Infrastructure: Build and configure secure, production-ready platforms for running large language models and other AI workloads across on-premises, private-cloud, hybrid, and restricted-network architectures. • Operate Model-Serving Platforms: Deploy, configure, qualify, and maintain models, inference runtimes, gateways, and supporting services throughout their operational lifecycle. • Optimize Performance and Capacity: Measure and improve latency, throughput, concurrency, resource utilization, and model placement using repeatable benchmarks and real workload evidence. • Build for Reliability and Security: Implement monitoring, alerting, access controls, auditability, incident response, recovery procedures, and secure software and model lifecycle practices. • Automate Platform Delivery: Use infrastructure-as-code, configuration management, testing, and release automation to make deployments, upgrades, scaling, and rollback repeatable and boring. • Partner and Transfer Knowledge: Work directly with client and application teams, communicate technical tradeoffs, document decisions and operating procedures, and leave engineers genuinely able to run and extend the platform. • Innovate How We Work: Experiment with and define the agentic workflows, automation, and tooling that improve how Nexera builds and operates AI platforms. Your patterns become part of our operating model. What We're Looking ForThe bullets below are the profile we hire against; the Tools and Technologies section that follows is preferred, not required. If you meet most of what is here and can show real production work, we want to hear from you. • Systems experience that shows. Infrastructure, Linux systems, SRE, platform engineering, HPC, distributed systems, MLOps, or production ML engineering. Roughly three to five years, or equivalent capability you can demonstrate. • Hands-on LLM and local-inference work. You have deployed or operated LLMs, inference runtimes, or GPU-backed workloads in production. • Strong Linux fundamentals. You are comfortable troubleshooting below the application layer: services, processes, networking, storage, permissions, resource limits, containers, drivers, and OS behavior. • Practical GPU knowledge. You understand GPU memory, model placement, multi-GPU communication, utilization, thermal and power behavior, and how model architecture, precision, context length, and batching translate into capacity. • Production ownership. You have run systems other people depend on and know what monitoring, alerts, incident response, upgrades, backups, rollback, and handoff actually require. • Performance discipline. You measure before you optimize, you state the workload behind your numbers, and you can walk someone through how you found a bottleneck across application, runtime, memory, communication, or hardware. • Security-minded engineering. Least privilege, network isolation, artifact integrity, identity propagation, auditability, vulnerability management, and restricted-network operations are familiar territory. • Daily AI tool user. You already use AI tools (Claude, Claude Code, Copilot, Cursor, or similar) in how you code, research, and solve problems, and you can clearly show and demonstrate how you apply them to day-to-day engineering work. • Relentless curiosity. You follow new model and runtime releases because you find it interesting, not because someone asked you to. You have opinions about where local inference is headed and you want to prove them out. • Strong communication skills. You can explain a GPU bottleneck to an engineer, summarize risk and options for a program lead, work constructively with security and data-center teams, and document decisions clearly. • Consulting mindset. You can work in a client’s environment, navigate ambiguity, surface blockers early, make pragmatic recommendations, and deliver against agreed outcomes. • U.S. citizenship and residency: Required for this role due to the customer environments and information the work may support. • Military or cleared experience preferred: Prior U.S. military service, experience supporting defense or national-security organizations, and/or a current or previously held U.S. government security clearance are preferred but not required. No clearance today is not a barrier: Nexera will sponsor clearance processing when assigned work requires it. Tools and Technologies (Preferred)You do not need every item below. Depth with one production inference stack, strong systems fundamentals, and evidence that you learn adjacent technologies fast are what we will weigh and evaluate. • Operating systems and hardening: RHEL, Rocky Linux, AlmaLinux, SELinux, FIPS mode, DISA STIG, OpenSCAP, systemd • NVIDIA GPU stack: H100/A100/HGX-class systems, CUDA, NCCL, NVLink/NVSwitch, Fabric Manager, NVML, DCGM/DCGM Exporter, Nsight Systems, Nsight Compute • Inference and ML runtimes: vLLM, SGLang, TensorRT-LLM, NVIDIA Triton or Dynamo, Hugging Face Transformers/TGI, PyTorch, Ray Serve • Inference concepts: tensor and pipeline parallelism, BF16/FP8/INT4, quantization, continuous batching, KV and prefix caching, prefill/decode behavior, speculative decoding, long-context serving, structured output, tool calling • Containers and orchestration: Podman, Docker, containerd, Kubernetes, OpenShift, NVIDIA GPU Operator, Slurm, Helm • Gateway and identity: NGINX, LiteLLM or comparable gateways, OpenAI-compatible APIs, Keycloak, FreeIPA, OIDC, JWT, service accounts, TLS • Observability and operations: Prometheus, Grafana, OpenTelemetry, Elastic, Wazuh, Splunk, Loki, alerting and SLO tooling • Automation and supply chain: Python, Bash, Ansible, Terraform or equivalent infrastructure-as-code, Git, private RPM/Python/container repositories, SBOMs, cosign, artifact signing and offline promotion • AI CLI and agentic tools: Claude Code, Gemini CLI, OpenAI Codex, GitHub Copilot • Additional differentiators: C++, CUDA or Triton kernel work, inference-runtime contributions, InfiniBand/RDMA/RoCE, regulated or classified environments, and on-premises or air-gapped deployments Position Details• Structure: Full-time or contract. We are flexible and will work out the right arrangement with the right person. • Location: Remote-first (US-based), with scheduled on-site work at client sites and data centers • Travel: Expect approximately 25% travel, concentrated in bursts around platform bring-up, qualification, deployment, and major troubleshooting events, with lighter travel between deployments. On-site work is scheduled in advance wherever possible. • Eligibility: U.S. citizenship and current U.S. residency are required • Access Requirements: Must be willing and able to complete applicable client background investigations, satisfy site-access and data-handling requirements, and obtain and maintain a U.S. government security clearance if required for assigned work • Compensation: Full-time base salary range: $175,000–$225,000, based on demonstrated experience, technical depth, clearance status, and scope of responsibility, plus performance bonus. Contract engagements: approximately $95–$150/hour (W-2) or $120–$175/hour (corp-to-corp), commensurate with experience, specialization, and clearance. Benefits (Full-Time)• Time off: Flexible PTO plus federal holidays • Learning: Annual learning and certification budget, plus access to premium AI tooling (Claude Code and the frontier tools you will use daily) • Home office: Home-office setup • Travel comfort: Sensible travel policy — you book what makes the trip productive, and travel time is respected as work time