Summary
✨ AI‑Generated
A data-intensive research organization is seeking a Senior Engineer to lead a web-scraping and data-engineering team. You will design and deploy advanced collection systems for novel, unstructured datasets, using AI/ML and agentic applications to build high-capacity, low-latency pipelines. The work may span web, audio, and video data and directly support sophisticated research workflows.
Highlights
Lead a team building advanced data-collection systems from diverse and unstructured sources. Apply AI/ML and agentic technologies to create high-capacity, low-latency systems that support data-driven research and investment decisions.
Description
Role Overview
We are seeking a technically proficient, creative, and detail-oriented Senior Engineer within our Webscraping team.
With thousands of novel and unstructured datasets being used on a daily basis in our investment strategies, this role is crucial in helping our portfolio managers find a competitive advantage.
You will manage and work alongside a team of data engineers to build advanced collection systems for BAM’s proprietary data efforts, drawing upon your passion of AI / ML to create high capacity, low latency systems that enable our researchers and analysts to gain deeper insights.
From webscraping, to audio, to video, your work will be a direct input to our firm’s investment decisions.
Team's Key Responsibilities
· Team leadership: Manage a team of talented, ambitious developers to optimize their impact and potential.
· Tool Development: Design, develop, and deploy Agentic applications to extract to collect and standardize unique data for our investment researchers and quants.
· Process Improvements: Analyze existing processes and identify areas where AI/ML can be applied to improve accuracy, speed, and efficiency.
· Technology Infrastructure: Work with technology and data teams to build and maintain robust technology infrastructure that supports these efforts.
· Performance Optimization: Continuously evaluate and refine models and algorithms to ensure optimal performance and alignment with business goals.
· Offshore Management: Partner with the firm’s offshore resources to scale human in the loop data validation and collection.
· Collaboration: Partner with portfolio managers, analysts, and other stakeholders to understand business needs and translate them into technical solutions.
· Research and Innovation: Stay abreast of the latest developments in technologies and identify innovative approaches to enhance our competitive edge.
Minimum Qualifications
Bachelor’s or Master’s degree in Computer Science, Engineering, Mathematics or a related field, preferably within tech company.
· 7-10+ years of software development experience.
· Proven experience managing software or data engineering teams.
· Strong proficiency in Python (including pandas) and SQL/Database skills.
· Experience with agentic coding (e.g., Claude Code, MCPs, skills, subagents).
· Experience with data pipelines (Airflow, Kafka).
· Experience with web technologies (HTML/JS, APIs) and networking.
· Strong understanding of data structures, algorithms, and statistical methods.
· Excellent verbal and written communication skills.
Preferred Qualifications
· Prior experience in multi-strategy hedge funds or similarly complex investment organizations.
· Experience designing infrastructure, data products, and tools for Analysts, Quants, and Data Scientists.
· Proficiency in containerizing workloads with Docker (Kubernetes experience is a plus).
· Familiarity with cloud technologies, particularly AWS.
Soft Skills:
· People leadership and coaching skills.
· Strong problem-solving and analytical skills.
· Excellent communication and interpersonal skills.
· Ability to work independently and as part of a team.
· Demonstrated ability to manage multiple projects and meet deadlines.
Even if your experience doesn’t match every requirement, we’d still love to hear from you—if you’re excited about this role, we encourage you to apply!
