Summary
Join an AI engineering team focused on ensuring the quality and trustworthiness of generative AI systems used in a demanding travel-related environment. You will develop and apply testing frameworks for LLMs and AI agents, evaluate model behavior, test prompts, detect hallucinations, and validate security, accuracy, reliability, and performance. Strong Python skills and substantial hands-on experience with generative AI, LLM evaluation, agentic systems, and QA automation are essential.
Highlights
Senior AI quality engineering opportunity focused on the reliability, accuracy, security, and performance of generative AI systems. The role provides hands-on work with LLMs, AI agents, evaluation frameworks, prompt testing, hallucination detection, and responsible AI, with relocation possible for suitable international candidates.
Description
Title: AI Engineer - Generative AI Testing (Airline/Travel Domain exp.
is a must)
Location : - Abu Dhabi (Onsite)
Exp.: 8-9+ Yrs.
Note: Looking for immediate joiners or candidates who are currently serving their notice period.
It is acceptable to consider candidates from India, provided they have a genuine interest in the opportunity and are willing to relocate.
Key Skills: Python | Generative AI | LLMs | AI Testing | LLM Evaluation | AI Agent Testing | Prompt Testing | Hallucination Testing | AI Agents | Agentic AI | Airline Domain | QA Automation | Responsible AI.
Role Overview
We are seeking an experienced AI Engineer with strong expertise in Generative AI, LLMs, AI Agents, and AI Testing.
The primary focus of this role is to ensure the quality, reliability, accuracy, security, and performance of AI-powered solutions used within the airline industry.
The ideal candidate will have hands-on experience developing AI applications and establishing robust testing frameworks for LLMs, AI agents, and conversational AI systems.
Key Responsibilities
Develop and support AI applications using Python, Generative AI, and Large Language Models (LLMs).Design, execute, and automate comprehensive AI test strategies across the AI application lifecycle.Validate AI models, LLMs, AI agents, and agentic workflows for accuracy, consistency, reliability, and safety.Perform functional, integration, regression, performance, security, and user acceptance testing for AI solutions.Conduct LLM evaluation, prompt testing, hallucination testing, bias and fairness assessments, and guardrail validation.Develop automated frameworks for testing AI agents, chatbots, copilots, and autonomous workflows.Define and track AI quality metrics, benchmarks, and acceptance criteria.Identify, analyze, and resolve defects related to AI outputs, workflows, and model behavior.Collaborate with business stakeholders and development teams to ensure production-ready AI solutions for airline operations.
Required Skills
Strong programming expertise in Python.Hands-on experience with LLMs, Generative AI, RAG architectures, Vector Databases, and AI Models.Experience with AI frameworks such as LangChain, LangGraph, CrewAI, AutoGen, LlamaIndex, or Semantic Kernel.Experience developing and testing AI Agents, Agentic AI solutions, Chatbots, and Conversational AI platforms.Strong expertise in AI Testing, including:LLM Evaluation & ValidationAI Agent TestingPrompt Testing & OptimizationHallucination TestingBias & Fairness TestingFunctional TestingIntegration TestingRegression TestingPerformance & Load TestingSecurity & Responsible AI TestingTest Automation for AI ApplicationsExperience creating AI quality frameworks, test cases, evaluation datasets, and benchmarking methodologies.Familiarity with Azure OpenAI, AWS AI, Google AI services, or equivalent platforms.
Domain Experience
Experience in the Airline, Aviation, Travel, or Transportation domain is required.Knowledge of airline operations, reservations, customer service, crew management, airport operations, or passenger experience solutions is an advantage.
Preferred Qualifications
Bachelor's or Master's degree in Computer Science, Artificial Intelligence, Data Science, Engineering, or related field.Certifications in AI, Machine Learning, Software Testing, or Cloud Technologies are preferred.
