AI Platform DevOps Engineer

Key Responsibilities Build and operate observability for LLM gateways, model routing, MCP servers, agent runtimes, and AI platform services.Implement end-to-end telemetry using OpenTelemetry, Langfuse, metrics, logs, and distributed tracing.Monitor and analyze LLM/agent latency, errors, token usage, model performance, quality, evaluation results, and production behavior.Develop dashboards, alerts, and reporting for AI reliability, performance, quality, and operational health.Build AI cost and usage observability, including attribution by application, team, user, model, workflow, and environment across providers such as OpenAI, Anthropic, and Gemini.Establish observability and evaluation standards for LLM and agentic applications, including traces, prompts, responses, tool calls, evaluations, and regression signals.Support AI evaluation and monitoring through golden datasets, scoring, prompt/model comparisons, and production quality checks.Operate AI platform services on Kubernetes using Helm, Kustomize, GitOps, autoscaling, and progressive/zero-downtime deployments.Implement platform security, access control, secrets management, PII protection, guardrails, and audit logging.Key Skills AI/LLM Observability: Langfuse, OpenTelemetry, LLM tracing, AI evaluationAI Platforms: LLM gateways, model routing, MCP, agentic AICloud/Platform: Kubernetes, Helm, Kustomize, GitOps, APIsMonitoring: Prometheus, Grafana, Datadog or similar platformsAI/ML: OpenAI, Anthropic, Gemini, LangChain, LangGraph, CrewAI, Google ADKEngineering: Python/Java/Go, REST APIs, CI/CDSecurity: OAuth2/OIDC, JWT, RBAC, secrets management, PII/AI guardrails