Senior Python Engineer - AI Agents & Data Discovery

Senior Python Engineer – AI Agents & Data Discovery About the Client Our client is a leading global investment management company headquartered in London, managing over $228 billion in assets and serving institutional investors, pension funds, wealth managers, and other sophisticated clients worldwide. The firm specialises in quantitative investing, alternative investments, systematic trading strategies, and technology-driven asset management. Data science, machine learning, and artificial intelligence are core components of its investment, research, and technology processes. As part of our collaboration, we are focusing on two foundational capabilities required to enable safe and scalable AI adoption across the enterprise: Agentic Security and AI-Ready Data Foundations. Role Overview We are looking for a Senior Python Engineer with strong AI/LLM and data-engineering experience to build the agent-facing discovery layer that enables AI assistants to safely find, understand, and retrieve trusted enterprise data. The role sits at the intersection of Python engineering, AI agents, LLM application development, semantic search, data catalogues, and financial data. The value of enterprise AI is ultimately limited by the data its agents can reach. If an agent cannot discover, interpret, trace, or correctly route a request to the appropriate data source, library, or report, the resulting capability is ineffective or potentially unsafe. In this hands-on senior role, you will build AI agent tools, skills, semantic routing, conversational discovery, retrieval, and evaluation capabilities on top of existing governed enterprise data catalogues and AI platforms. You will work within the client's existing AI platform, gateway, marketplace, and governance framework rather than creating a parallel platform. Key Responsibilities Build the agent-facing discovery layer over domain data catalogues.Develop AI agent skills and tools that enable the client's AI assistant to:Discover datasetsFind market-data sourcesIdentify relevant symbols and fieldsDiscover curated reportsExplain available dataRetrieve appropriate informationImplement semantic routing from natural-language user questions to the correct:Data libraryDatasetMarket-data symbolFieldReportImplement step-by-step candidate narrowing using inexpensive lookups rather than hard-coded, source-specific integrations.Build hybrid retrieval approaches combining lexical and semantic matching.Develop guided conversational discovery capabilities, including:Clarifying questionsCandidate suggestionsExplanations of candidate selectionsIntent refinementCatalogue-grounded conversationsBuild and integrate agent tools and skills using MCP servers or equivalent tool-integration frameworks.Develop packaged skills/plugins and publish them through the client's central AI assistant platform.Integrate with governed platform APIs while following established rules for:ProvenanceCitationsFreshnessSecurityGovernanceCo-design governed lookup capabilities with client platform teams, such as agent-consumable report and dataset discovery APIs.Publish and maintain AI skills through the client's marketplace and review process.Manage skill versioning, evaluation coverage, and staleness.Build automated evaluation frameworks for AI discovery and routing quality.Convert representative user questions into automated acceptance and regression tests.Gate releases based on routing correctness and answer-quality regression results.Monitor real-world usage and feed discovery failures back into the metadata layer.Identify:Failed routesAmbiguous terminologyMissing descriptionsIncomplete metadataIncorrect candidate suggestionsTranslate these findings into concrete catalogue and metadata improvements.Work closely with client engineering, data, AI, and platform teams to present designs, incorporate feedback, and deliver production-quality solutions.Help establish engineering standards for reliable, testable, and governed AI-agent applications. Required Skills & Experience Python Engineering 5+ years of production software development experience in Python.Strong software engineering fundamentals, including:TestingCode qualityCode reviewPerformanceMaintainabilityProduction supportLLM Application Development 2+ years of hands-on LLM application development.Strong practical experience with:Tool/function callingStructured outputsContext managementPrompt engineeringLLM application orchestrationExperience building production-oriented applications using LLMs rather than only experimental or proof-of-concept solutions.AI Agents & Tool Integration Experience building AI agent tools and skills.Hands-on experience with MCP servers or equivalent agent/tool integration frameworks.Experience packaging and publishing skills/plugins through a central AI assistant or managed AI platform.Understanding of how AI agents discover and invoke tools to complete user requests.Retrieval & Semantic Routing Experience implementing retrieval and routing over structured metadata.Ability to route natural-language questions to the correct data source, library, dataset, symbol, field, or report.Experience with:Semantic routingCandidate retrievalHybrid lexical/semantic searchMetadata-based filteringStep-by-step candidate narrowingConversational Discovery Experience designing guided conversational discovery experiences.Ability to build agents that:Ask appropriate clarifying questionsSuggest candidatesExplain why candidates were selectedRefine user intentGround responses in trusted catalogue metadataGoverned AI Platforms Experience building against governed platform APIs.Experience publishing AI capabilities through a managed review and governance process.Understanding of requirements around:ProvenanceCitationFreshnessVersioningCompliancePlatform behaviour rulesAI Evaluation Strong evaluation discipline for LLM and agent applications.Experience creating automated test suites for:Routing correctnessRetrieval qualityAnswer qualityRegression testingAbility to establish release gates based on evaluation results.Communication Fluent English, both written and spoken.Comfortable working independently with client engineering teams.Ability to present technical designs, respond to feedback, and represent the delivery team in technical discussions.Nice to Have Experience with financial services or other regulated environments.Knowledge of symbology and instrument reference data, including:Identifier regimesEntity resolutionMapping company names to internal identifiersKnowledge of fund or strategy reporting and BI.Experience co-designing governed lookup APIs with platform teams.Production experience with:EmbeddingsVector searchRerankingExperience optimising AI agent workflows for:Token efficiencyLatencyCostExperience consuming knowledge graphs at query time.Experience with LLM-as-a-judge evaluation frameworks.Day-to-day experience using AI coding agents.Strong judgement around conversational UX, particularly determining when an agent should ask for clarification versus make a reasonable assumption.Client-facing engineering experience.Technical Skills Must Have PythonLLM Application DevelopmentAI AgentsTool / Function CallingStructured OutputsMCP or Equivalent Agent Tool FrameworksSemantic RoutingMetadata RetrievalHybrid SearchConversational DiscoveryAI Evaluation & Regression TestingGoverned AI PlatformsAPI IntegrationNice to Have EmbeddingsVector SearchRerankingKnowledge GraphsLLM-as-a-JudgeFinancial/Market DataSymbologyReporting & BIAI Coding AgentsRegulated Financial ServicesEducation Bachelor's degree in computer science, Software Engineering, Data Science, Artificial Intelligence, Information Technology, or a related technical discipline. Why This Position? This role sits at the intersection of AI engineering, data engineering, agentic systems, and financial services, addressing one of the most important challenges in enterprise AI: enabling agents to securely discover and reason over trusted enterprise data. You will have significant technical ownership and the opportunity to build foundational capabilities that directly shape how AI assistants interact with enterprise data. The position provides exposure to modern AI agent architectures, LLM applications, semantic retrieval, governed AI platforms, data catalogues, and evaluation frameworks within a sophisticated and highly regulated investment-management environment. Most importantly, you will help establish the foundations that allow an organization to move from AI experimentation to safe, scalable, production-grade agentic AI.