Device Lab and Benchmark Engineer

Every model we ship needs latency, memory, and accuracy numbers from real devices. Build the lab and the tools that produce those numbers. About Desert Ant Labs Desert Ant Labs is an on-device AI lab in Europe. We're building efficient frontier intelligence, bottom up. We make the fastest model for each task, built to run on the billions of phones, laptops, and browsers people already own. Every model is lightning fast and private, and has no per-call cost. Little brains in every product. With our native SDKs for Swift, Kotlin, and JavaScript, you add a model to an app in a few lines of code. Data stays with your customer, and the feature works without a cloud provider. Every role here pushes the limits of what a phone or a laptop can do. About the role You work on the device lab and the benchmarks we run there. The lab is dozens of phones, tablets, and laptops connected to machines in our Amsterdam office, and the team has more devices across Europe and North America. You build the system that runs a model on every device in the lab. Anyone on the team should be able to request a benchmark with one command and get the same result a week later. Go-to-market publishes those results. The goal is numbers that show what each device can do, so we know how far to push each model. Customers compare our models by the benchmarks we publish. A phone runs a model more slowly when the phone is warm or charging, so the lab controls for temperature, charging, and background load before a number goes public. You also report where a model runs slowly, with a profile and a proposed fix for the model and runtime engineers. You test our models on hardware we don't support yet, including new phones and robots. You build demo apps with go-to-market and marketing that show a model running on a specific device. What you will do Build and run the device lab: provisioning, flashing, pinned OS versions, and replacing devices that fail.Build a distributed benchmark system that sends jobs to devices in the lab and on the team, then collects and stores the results.Run benchmarks in CI, so a performance regression fails the build before a release.Keep an inventory of all team hardware: laptops, phones, tablets, and training machines.Send bottleneck reports to the model and runtime engineers, with a profile and a proposed fix.Build demo apps and examples with go-to-market and marketing.Turn results into charts, model card sections, and blog posts.Run our models on new hardware, from the latest phones to robots. You might be a fit if you Have built internal tools that other engineers used every day.Have run a device lab or device farm, or can describe in detail how you would run a device lab.Can administer macOS and Linux machines and the phones and tablets connected to those machines.Know what skews a mobile benchmark: thermal throttling, background load, warm versus cold runs, and charging state.Have measured models on Core ML, LiteRT, or ONNX Runtime and can explain why results vary between runs.Live in or near Amsterdam, where the lab is.Write clean, tested code, and spot a weak change in review, whether a person or an agent wrote it.Plan and run your work through coding agents such as Claude Code, with a low tolerance for slop. Bonus points Built a device farm or test lab at a mobile or hardware company.Wrote a benchmark post that others cited.Worked with single-board computers, robotics, or embedded Linux. What we offer Salary and equity, based on level.In our Amsterdam office. How we work We're a small, flat team, and we build the models, the SDKs, and the apps that use them. Detail and Subwave, our own apps, run our models in production. Start what needs starting without waiting to be asked, and finish what you start. Take on work outside your role when a project needs you. We ship quickly, so we cut scope until only the part users notice is left. Anyone can comment on your work or redo your draft, and we say early when work isn't ready. We read that feedback as help. We judge the work by what shipped and what changed because of it. Nobody counts hours. Recruiters and agencies We hire directly. We don't reply to emails from recruiters or agencies, and we don't want their outreach about this role or any other. Location: Amsterdam Apply on our site: https://desertant.com/jobs/device-lab-and-benchmark-engineer/