Quality Assurance Automation Engineer

QA Engineer — AI-augmented Test AutomationFull-time · Hybrid (Yerevan) · Essential · essentialsln.com Essential curates small teams of world-class engineers to accelerate our clients' innovation cycles. For this role you will own product quality for a production AI companion: users talk to an AI avatar via video, voice, or text, powered by a self-hosted LLM stack, Supabase, Stripe billing, a React/TypeScript frontend, and an iOS app. This is a builder QA role, not a manual-execution role — most execution is automated or delegated to AI agents, and you build that automation. Stack: Playwright · TypeScript · Vitest / Deno test · GitHub Actions · API testing · SQL · LLM quality evals · AI test agents · i18n (DE primary). Your mission Own quality across text, voice, video, billing, and iOS — by building the automation that makes releases self-verifying, and by running AI test agents as your force multiplier for per-ticket feature verification. What you will do Own the testing roadmap • Close the business-logic gap: introduce Vitest for the logic-dense frontend modules (entitlement/free-trial math, token balance and cost calculations, crisis-UI state) and per-handler Deno tests for the streaming edge functions with mocked LLM fixtures (split-chunk buffering, output filters, auth rejection, token debits). • Run the AI feature-test agent: for each ticket merged to dev, have an agent read the ticket + attachments + diff, produce a 5–10 point test plan, execute it via browser automation with a dedicated test account, and post evidence to Linear. Codify the prompt as a skill with guardrails (dev only, test account only, never live payments, never destructive actions). Graduate repeatable checks into the permanent regression suite. • Put LLM quality monitoring on a schedule: run the judge-scored quality canary weekly, run the RAG variant on prompt/model changes, and persist scores so drift is visible over weeks, not just per run. Maintain and extend the regression suite • Playwright browser tier (real login, text chat E2E, spoken voice and video sessions with fake-mic audio), HTTP tier (liveness, auth-gate negatives, AI first-message canary, safety blocks, prompt-injection probes), and the manual release checklist (OAuth, Stripe, crisis UX, i18n across 6 locales, iOS/OTA, admin). • Every shipped feature gets its regression line; every escaped bug gets a test. Be the quality voice in the ticket flow • Review acceptance criteria before tickets are picked up; flag ambiguity early. • Read agent feature-test reports critically and advise on promotion to production — agents inform, humans decide. • Own release verification and the post-release manual tier (~30 min today; your job is to shrink it). Must-have • 3+ years in QA / test automation, with real code ownership of test suites in TypeScript/JavaScript. • Playwright (or equivalent: Cypress, WebdriverIO) in CI, including flaky-test diagnosis and stabilisation. • Hands-on daily use of AI agents for testing — you will be asked to show this live. • Comfortable with API testing, SQL for read-only data verification, reading logs, and GitHub Actions. • Structured thinker who writes concise, evidence-backed reports (screenshots, repro steps, logs) that engineers act on without a follow-up call. • Fluent written English; German strongly preferred (DE is the primary UI locale and needs a native-level eye in i18n review). Nice-to-have • Testing LLM-based products: eval design, judge models, rubric scoring, strategies for non-deterministic output, safety/red-team probing (prompt injection, crisis-scenario coverage). • Testing real-time media (WebRTC audio/video, fake media devices), voice UX. • Vitest / Deno test, Supabase, Stripe test mode, iOS TestFlight / Capacitor · accessibility and i18n testing. • Experience in health or other safety-sensitive domains. Working principles Prioritisation rule: automation and agent leverage first, manual execution last. Manual work that recurs three times must become a script, a Playwright spec, or an agent skill. Success after 90 days • The previously untested entitlement and token-math logic runs in CI with focused unit suites. • The AI feature-test agent has run on 5+ tickets with evidence-backed reports, its prompt codified as a skill, and at least two discovered checks graduated into the regression suite. • The post-release manual tier has measurably shrunk; the LLM quality canary runs on a schedule with a baseline. What we offer • Hybrid, async-friendly, small team with direct product impact. • A mature testing and agent setup to build on: Playwright + HTTP regression tiers, CI, MCP tooling, and eval scripts already exist. • Real ownership of the quality roadmap — you decide what gets automated next. • Budget for AI tooling (Cursor, Claude, model APIs) is part of the job, not a perk. How to apply Send your CV plus a short note on how you have used AI agents in testing to jobs@essentialsln.com. Full description: https://www.essentialsln.com/careers/qa-engineer