Summary
✨ AI‑Generated
As a Senior Software Engineer, you will be among the early engineers establishing a new capability focused on training and evaluating AI agents. You will build scalable infrastructure that tests agent performance against real human expertise in secure environments, while defining technical direction and architecture in a highly ambiguous 0-to-1 setting.
Highlights
A genuine 0-to-1 senior engineering opportunity with substantial influence over technical direction, architecture, and product development. The role involves building infrastructure for evaluating AI agents at scale in secure environments and working closely across engineering, product, and AI research.
Description
Senior Software Engineer - (Agent Training and Evals)
Engineering
Prolific
Prolific is not just another player in the AI space – we are the architects of the human data infrastructure that's reshaping the landscape of AI development.
In a world where foundational AI technologies are increasingly commoditized, it's the quality and diversity of human-generated data that truly differentiates products and models.
The role
As a Senior Software Engineer, you'll be one of the first engineers on our Agent Training & Evals team.You'll build infrastructure to evaluate AI agents at scale against real human expertise, in safe and secure environments.
The better we get at this, the more useful, reliable and safe agents become.This is a genuine 0→1 role.
There isn't a well-established playbook or existing architecture for you to simply maintain.
You’ll shape the team’s technical direction and the products we build.You'll work closely with engineering, product and AI research to turn ambiguous problems into scalable systems, and you'll see the impact of what you build directly.
What you’ll be doing in the role
Build a platform to support evaluating and fine-tuning agents Take prototypes to production; turn experiments and research ideas into reliable, well-tested softwareOwn what we ship as it scales in productionSolve hard problems around data, compute, orchestration, scaling, security and reliabilityWork with emerging agentic AI technologies and rapidly evolving modelsMake effective product and technical decisions in a highly experimental environmentPartner closely with customers, product engineering and AI research to turn problems and opportunities into working productsDrive direct impact in a fast-moving, high-ownership, cross-functional team
What you’ll bring to the role
A background in Software Engineering, Data Engineering or MLOpsStrong experience building scalable infrastructure or distributed systems in the cloud Solid understanding of how systems are deployed, scaled and securedExperience applying good engineering practice and raising the bar for those around youHands-on experience with integrating and evaluating AI models or agentsInterest in how AI agents are trained, evaluated and improvedA product mindset with a focus on delivering business and customer valueComfort with ambiguity, and the ability to move quickly from idea to working systemPython experience is useful but not essential
Key Technologies
Cloud Platforms: Google Cloud PlatformProgramming Languages: Python, JavaScript, and TypeScriptFrameworks: FastAPI, Vue.js, Container-based and Serverless architecturesDatabases: PostGres SQLDevOps and Monitoring: GitHub Actions, Kubernetes, Datadog, Langfuse
Why Prolific is a great place to work
We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioral data and feedback.
This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.
We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviors into AI development.
By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation – one that reflects the breadth and the best of humanity.
Working for us will place you at the forefront of AI innovation, providing access to our unique human data platform and opportunities for groundbreaking research.
Join us to enjoy a competitive salary, benefits, and remote working within our impactful, mission-driven culture.
Links to more information on Prolific
Benefits
External Handbook
Website
Youtube
Privacy Statement
By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning.
Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal information.
