Agentic AI Quality Engineering Lead

Agentic AI Quality Engineering Lead Must be located in Quebec. ROLE FOCUS: Production quality engineering for Salesforce Agentforce and AI-enabled Salesforce solutions DELIVERY POSTURE: Hands-on automation architecture, AI evaluation, security testing and release assurance PRIMARY PLATFORM: Agentforce, Data Cloud, Experience Cloud, Service Cloud and the Salesforce integration ecosystem CORE TOOLING: Playwright with TypeScript, Apex tests, LWC Jest, API automation, CI/CD and AI evaluation frameworks About the role As an Agentic AI Quality Engineering Lead, you will own the end-to-end quality strategy for Salesforce Agentforce implementations. You will establish the automation, evaluation, security, accessibility, evidence and release controls required to move AI agents and Salesforce capabilities into production with confidence. This is not a traditional manual test-management role. You will combine Salesforce quality engineering with agent evaluation and AI-assisted test automation, working closely with Forward Deployed Engineers, Salesforce architects, business analysts, security specialists and client teams. Salesforce context: This role is designed specifically for Salesforce and Agentforce delivery, not as a platform-agnostic AI position. What you will do • Own the quality strategy across Agentforce agents, Apex, Lightning Web Components, Flows, Data Cloud, Experience Cloud, Service Cloud, APIs, middleware and external tools. • Design a layered automation architecture covering unit, component, API, integration, UI, end-to-end, security, accessibility and production-validation testing. • Build and maintain Playwright frameworks in TypeScript using reusable fixtures, page or component abstractions, stable locators, test-data utilities, tagging, parallel execution and environment-aware configuration. • Use Playwright traces, screenshots, videos, console logs and network evidence for diagnosis, auditability and defect reproduction. • Apply Playwright planner, generator and healer patterns to accelerate test creation and maintenance, while retaining human approval for authoritative regression assets. • Convert approved user stories, acceptance criteria, Salesforce personas and permission models into executable regression suites. • Integrate automation into GitHub Actions, Azure DevOps, Jenkins or equivalent pipelines, with risk-based pull-request and deployment gates. • Create Apex unit and integration tests, LWC Jest tests, Flow validation, API and contract tests, and Playwright end-to-end tests. • Design Agentforce evaluations for topic classification, instruction adherence, action selection, tool invocation, grounding, response quality, fallback and escalation behaviour. • Lead prompt-injection, jailbreak, indirect-injection, data-leakage, unauthorized-action and privilege-escalation testing. • Validate Salesforce security controls including CRUD/FLS, sharing, permission sets, user-mode execution, Named Credentials, External Credentials, data masking and Trust Layer controls. • Define production-readiness criteria, maintain requirements-to-test traceability and provide evidence-based release recommendations. • Implement production observability for task completion, fallback, escalation, latency, tool failures, grounding failures and policy violations. • Contribute reusable test skills, validators, datasets, automation components and quality gates to IBM Salesforce delivery assets. Required skills and experience Salesforce quality engineering • 6+ years in software quality engineering or test automation, including at least 3 years testing Salesforce implementations. • Strong working knowledge of Apex, LWC, Flows, SOQL, Experience Cloud, Service Cloud, Agentforce, Data Cloud, Salesforce security and integration patterns. • Ability to test Lightning UI behaviour, asynchronous rendering, dynamic identifiers, permission-based layouts and Experience Cloud journeys. • Hands-on experience with Apex test classes, test-data factories, mocking or stubbing, bulk and negative testing, and asynchronous processing. • Hands-on experience with LWC Jest testing and accessibility validation. • Experience testing REST/SOAP APIs, Platform Events, middleware, event-driven processes and external services. • Experience with sf CLI v2, source-driven delivery, Salesforce environments, Git workflows and CI/CD. Automation engineering • Expert-level Playwright using TypeScript, including fixtures, projects, storage state, locator strategy, assertions, parallelism, retries, trace viewer and reporters. • Ability to build maintainable UI automation without brittle selectors or uncontrolled self-healing. • Experience with Jest and an API automation approach such as Postman/Newman, REST Assured, pytest or equivalent. • Experience with contract testing, service virtualization, test doubles and deterministic test-data management. • Ability to define execution tiers such as pull-request smoke, nightly regression and pre-release certification. • Experience publishing machine-readable evidence and enforcing CI/CD quality gates. • Strong coding and root-cause analysis across browser, Salesforce, API, integration and data layers. Agentic AI evaluation and security • Hands-on experience testing LLM-powered applications, AI assistants, Agentforce agents or autonomous workflows. • Understanding of probabilistic evaluation, benchmark datasets, repeated trials, rubric-based scoring and human review. • Ability to assess groundedness, relevance, completeness, instruction adherence, task completion, action correctness and safe refusal. • Experience with Promptfoo, DeepEval, Braintrust, LangSmith or comparable enterprise evaluation tooling. • Understanding of RAG evaluation, source authority, stale or conflicting content and permission-aware retrieval. • Experience testing prompt injection, jailbreaks, sensitive-data exposure, unsafe tool use and cross-user data access. • Familiarity with multi-agent orchestration, tool or function calling, MCP and model routing. Leadership and delivery traits • Ability to define a practical QE architecture while remaining hands-on in framework development and critical automation. • Executive-ready communication of quality status, residual risk and release recommendations. • Ability to lead defect triage and verify root cause and closure evidence. • Comfort operating in rapid, ambiguous client environments and partnering closely with engineering teams. • Quality discipline across secure coding, documentation and WCAG 2.1 AA accessibility. Qualifications • Bachelor's degree in Computer Science, Engineering, Information Systems or equivalent practical experience. • 7+ years in quality engineering, test automation or software engineering in test. • 3+ years of Salesforce delivery experience. • Track record implementing automated quality controls within CI/CD and supporting production releases. • Consulting or client-facing delivery experience preferred. Certifications Required, or within 3 months of hire • Salesforce Certified Agentforce Specialist • Salesforce Certified Platform Developer I Strongly preferred • Salesforce Certified Platform App Builder • Salesforce Certified Data Cloud Consultant • Salesforce Certified AI Associate • ISTQB Test Automation Engineer, Advanced Test Manager or equivalent Assets • Platform Developer II or Application Architect • JavaScript Developer I or Integration Architect • Public Sector Solutions and Government Cloud experience • Bilingual English/French for Canadian public sector engagements • Eligibility for Canadian government Reliability Status or Secret clearance What success looks like • Every release has traceable automated evidence across Salesforce functionality and agent behavior. • Authoritative regression tests are human-approved, version-controlled and automatically executed at appropriate delivery gates. • Agent evaluations detect quality, grounding, security and behavioral regressions caused by prompt, model, data or tool changes. • Testing operates continuously across development, deployment and production monitoring. • Reusable QE assets improve consistency and accelerate future Salesforce Agentforce deliveries