Tech Lead, Evals Engineering

AboutBanks, payments companies, and fintechs are putting AI agents in front of millions of customers, but there's still no accepted way to show those agents follow the rules. We're an early-stage AI startup building the risk and regulatory measurement, so these companies can prove their agents are compliant and trust them with more of the work. Why joinA compelling unsolved problem. How to measure AI agent risk in high stakes regulated contexts is being worked out right now. The standards set in the next few years will shape how enterprises use AI for a long time, and you'd help define them.Real ownership. You'll lead engineering, building both the product and the team behind it with collaboration with the founders.Team with background in evaluation science, regulatory standards and enterprise AI. You'll work closely with founders who bring AI risk and financial regulation, machine learning research, and production AI systems, plus advisors with deep banking and regulatory experience.Upside. A competitive salary and early-team equity in a company at the start of its growth. What you'll doOwn the product's engineering end to end: architecture, technical roadmap, timelines, and the quality of everything we ship.Hire and lead the engineering team: define the roles, run hiring, and manage and grow the engineers who join.Set how we build: our practices for code review, testing, versioning, and releasing evaluations.Stay hands-on: design and build the backend, data pipelines, and evaluation systems that run over large volumes of real agent conversations and actions.Make our evaluations trustworthy: measure their accuracy, including precision, recall, agreement with expert reviewers, and drift over time, and improve it with our regulatory and domain experts.Deploy with customers: lead the technical side of pilots, and turn what you learn into product decisions. What we're looking for6+ years building production software, including time leading engineers as a tech lead or manager with prior startup experienceExperience hiring engineers and setting roadmaps and timelines for a team.Experience shipping LLM-based systems to production and knowing how to test and monitor their behavior.Strong backend and data engineering skills, and comfort working with large datasets and pipelines.Statistical rigor: sampling, inter-rater agreement, confidence intervals, and careful error analysis.Experience in banking, payments, or fintech Ownership and comfort with ambiguity at an early-stage company. Nice to haveExperience building evaluation infrastructure.Experience as an early engineer or engineering lead at a startup.Publications or open-source work in ML evaluation, benchmarking, or AI safety, risk, compliance Compensation and how to applyThe base salary range is 200-220K USD, plus an early-team equity grant. Final offers depend on experience and scope. To apply, email with subject “Tech Lead” and send following to jobs@mezurolabs.com Your resumeGitHub LinkedIn (if you have it) Short note on the hardest system you've owned end to end and what you would focus on in your first 90 days building out the engineering team. We are an equal opportunity employer. We welcome applicants of all backgrounds and do not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, veteran status, or any other legally protected characteristic.