Description
AI Engineers, Software Engineers & MCP Evaluation Experts.
Remote AI Project
We are seeking experienced AI Engineers, Software Engineers, Systems Engineers, Backend Engineers, ML Engineers, and Developer Tooling Experts to build reinforcement learning environments that test how advanced AI models solve complex software engineering problems using Model Context Protocol (MCP) tools.
The core of the role is strong software engineering.
Prior professional AI experience is helpful but not required.
What You’ll Do
Build reinforcement learning environments for software engineering tasksDesign realistic coding problems based on production-style codebasesCreate tasks involving bug fixing, feature implementation, refactoring, and optimizationRequire AI agents to discover information using real MCP tools and serversDesign scenarios where agents must inspect repositories, documentation, APIs, databases, logs, or other connected systemsCreate deterministic verification systems that accurately determine whether a solution is correctBuild automated tests and graders for coding tasksDevelop golden reference solutionsDefine clear success and failure conditionsEnsure environments are reproducible across repeated runsIdentify shortcuts or unintended ways agents could bypass task requirementsTest whether tasks measure genuine software engineering abilityEvaluate model behavior when using MCP toolsDebug environment, tooling, and infrastructure issuesImprove task difficulty, realism, and evaluation reliabilityDocument environment behavior, requirements, and expected solutionsCollaborate asynchronously with engineers, researchers, and reviewers
Who Can Apply
Relevant backgrounds include:
AI Engineers, Software Engineers, Senior Software Engineers, Backend Engineers, Systems Engineers, Platform Engineers, Infrastructure Engineers, Full-Stack Engineers, and Developer Productivity Engineers.
We also welcome:
Machine Learning Engineers, ML Systems Engineers, AI Infrastructure Engineers, Applied AI Engineers, Research Engineers, LLM Engineers, Agent Engineers, and AI Platform Engineers with strong software engineering fundamentals.
Systems and infrastructure backgrounds may include:
Distributed Systems Engineers, Cloud Engineers, DevOps Engineers, Site Reliability Engineers, Production Engineers, Performance Engineers, Reliability Engineers, and Infrastructure Software Engineers.
Developer tooling backgrounds may include:
Developer Tools Engineers, Build Engineers, CI/CD Engineers, Release Engineers, Test Infrastructure Engineers, Automation Engineers, Internal Tools Engineers, and Engineering Productivity Engineers.
Additional relevant backgrounds include:
Open Source Engineers, Compiler Engineers, Database Engineers, API Engineers, Integration Engineers, Security Engineers, Networking Engineers, Storage Engineers, and Runtime Engineers with strong coding and debugging experience.
Requirements
Strong professional software engineering experienceProficiency in at least one of C++, Python, Java, Go, TypeScript, or RustDeep understanding of algorithms and data structuresStrong debugging skillsExperience implementing production software featuresExperience refactoring existing codebasesAbility to optimize software for performance and scalabilityStrong understanding of testing and verificationAbility to work effectively in unfamiliar codebasesExcellent written and verbal communicationStrong attention to detailAbility to work independently in a remote environmentExperience collaborating across engineering teams
Preferred Background
Experience working on large-scale or distributed software systemsExperience with MCP, tool calling, agents, or AI coding systemsExperience building developer tools or automation infrastructureExperience creating coding benchmarks or evaluation environmentsExperience building automated gradersExperience with sandboxing or containerized environmentsExperience designing deterministic testsExperience with performance engineeringExperience working on open-source projectsExperience reviewing complex pull requestsExperience maintaining large production codebasesFamiliarity with machine learning or AI systemsExperience creating engineering best practices or technical standards
This opportunity is ideal for engineers who can take a real software engineering problem and turn it into a reproducible, challenging, automatically verifiable environment that tests whether an AI agent can actually debug, reason, use MCP tools, and modify a complex codebase correctly.
We are a referral partner of the client.