Senior MLOps Software Engineer

Your daily tasks: Design, build, and own the software solution used to assess cyber capabilities in AI models.Define a measurable benchmark and evaluation methodology covering relevant cyber-capability scenarios, tasks, scoring criteria, thresholds, and controls.Implement repeatable, automated evaluation pipelines that execute benchmark tests and generate consistent model-level results.Analyze model outputs, quantify performance, investigate failure modes, and translate findings into clear technical evidence.Partner with Model Risk Management, cyber security, data science, engineering, and governance stakeholders to align the approach with review expectations.Document evaluation assumptions, scope, data sources, test cases, metrics, limitations, versioning, and decision rationale to support independent challenge and auditability.Build reliable software components, APIs, data-processing workflows, and reporting outputs using secure engineering practices.Integrate evaluations into CI/CD and MLOps workflows, including experiment tracking, reproducibility, monitoring, and controlled release processes.Maintain benchmark relevance as model capabilities, cyber risks, and evaluation techniques evolve.Provide technical leadership, conduct code and design reviews, and drive the solution from proof of concept through production ownership. Your attitudes we value 7+ years of professional experience in software engineering, MLOps, machine learning engineering, data engineering, or a closely related discipline.Strong software engineering skills, preferably with Python, including modular design, testing, version control, APIs, and production-quality coding standards.Hands-on experience evaluating machine learning or generative AI model performance using quantitative and qualitative methods.Strong data analysis skills, including metric design, experiment analysis, error analysis, statistical reasoning, and clear visualization or reporting of results.Practical experience developing automated model evaluation or benchmarking frameworks and converting evaluation outcomes into decision-ready evidence.Experience interfacing with Model Risk Management, independent validation, risk, compliance, or governance teams.Working knowledge of MLOps practices such as CI/CD, model and dataset versioning, experiment tracking, reproducible execution, monitoring, and deployment controls.Ability to own a technical solution end to end, from requirements and architecture through implementation, validation, documentation, deployment, and support.Strong written and verbal communication skills, with the ability to explain complex evaluation results to technical, risk, and non-technical stakeholders. What we offer: Hybrid working model (2 days in the office from our Warsaw office, 3 days remote per week)Competitive salary with annual salary reviewsOpportunity to use foreign languages and the newest technologies on a daily basisIndefinite period employment contract after probation periodAccess to MyBenefit platform where you can choose from a variety of benefits (Sport, Tourism, Culture, Recreation etc.)Sport's card (Multisport)Private medical care for you and your familyPrivate life insuranceAccess to wide range of learning & development platformsDiscounted theatre ticketsWork in a multicultural environment