Senior Data Engineer

This role is primarily focused on data engineering, transformation frameworks, orchestration, and system integrations. While the team builds applications on top of their platform, dedicated application engineering resources lead full-stack UI development. This position will focus on building and maintaining reliable data pipelines, implementing business logic, and supporting scalable data solutions that power those applications. The ideal candidate has strong hands-on experience with modern cloud data platforms (Snowflake and/or Databricks), transformation tooling (dbt), and orchestration frameworks (Airflow or similar). This individual works effectively in a business-aligned environment, partnering with investment and operations teams to deliver secure and scalable data solutions. Primary Responsibilities Credit Data Warehouse Architecture & Development (40%) Build and enhance scalable data models within Snowflake (and/or Databricks where applicable) across landing, integration, and presentation layers, following established architectural patterns.Develop and maintain transformation logic using dbt, ensuring modular, testable, and well-documented modelsOptimize SQL performance and warehouse resource usage for large-scale financial datasetsImplement data quality checks, validation rules, and audit controlsContribute to metadata-driven approaches that support flexible integrations and reporting needs.Support lineage, governance, and maintainability of CDW assets through documentation and adherence to engineering standards.Design data models optimized for reporting and BI consumption, partnering closely with the Credit IQ (Power BI) team to ensure scalable semantic layers and performant analyticsWorkflow Orchestration & Pipeline Engineering (25%) Build and support data pipelines orchestrated through AirflowDevelop and maintain DAGs/workflows for ingestion, transformation, external extracts, and API integrations.Support improvements to pipeline reliability, monitoring, and error handling in production environments.Collaborate with DevOps to ensure CI/CD alignment and production stabilitySupport modernization of legacy orchestration processes into Airflow-based frameworksIntegrations & Data Products (20%) Build and support integrations with external vendors, fund administrators, trustees, and internal systems.Build scalable export frameworks (SFTP, API, file-based extracts) driven by configuration and metadataSupport data consumption by internal applications and analytics toolsCollaborate with application engineering teams to provide well-structured datasets and data interfaces for application use.Applied AI & Intelligent Data Use Cases (10%) Support AI-enabled workflows by preparing structured and unstructured data for retrieval and analysis use casesAssist in enabling retrieval-based workflows leveraging CDW datasets where applicableEnsure AI-related datasets follow security and governance standardsCross-Functional Collaboration & Technical Leadership (5%) Participate in code reviews and provide guidance to junior and offshore contributorsCoordinate assigned work with offshore resources to ensure clarity of requirements and timely deliveryWork with investment and operations teams to translate business workflows into scalable data solutionsPromote clear documentation and adherence to engineering best practicesRequirements Education & Certificates Bachelor's degree, requiredConcentration in computer science, engineering, or a related quantitative field, preferredMaster's degree preferredProfessional Experience Minimum of 6 years of overall relevant technical experience, requiredExperience in data engineering, platform engineering, or backend-focused software engineering roles, requiredStrong hands-on experience designing and operating solutions in Snowflake and/or Databricks (or comparable modern cloud data warehouse/lakehouse platforms)Experience building and deploying data platforms in Azure and/or AWS cloud environmentsStrong experience writing and optimizing SQL for analytical and financial datasetsHands-on experience with dbt for transformation management, testing, and modular model developmentExperience developing and supporting Airflow-based orchestration pipelines (or equivalent), including workflow design and monitoringExperience building and maintaining enterprise-grade data pipelines across ingestion, transformation, and presentation layersExperience integrating enterprise systems via APIs, file-based transfers (SFTP), or event-driven workflowsExposure to financial services, alternative asset management, or credit products strongly preferredExperience collaborating with offshore or distributed engineering teams preferredExperience supporting reporting and BI environments (e.g., Power BI), including building reporting-friendly data models and optimizing warehouse performance for analytics workloadsPython experience for pipeline tooling, automation, and integration servicesExperience with Azure services (e.g., Azure Data Factory, Azure Storage, Azure AD) and/or AWS services (e.g., S3, IAM, Lambda, ECS)Experience building or supporting data services consumed by applicationsExposure to applied AI or LLM-enabled workflows in a production settingFamiliarity with Power BI or comparable BI/reporting tools and semantic modeling conceptsStrong SQL and analytical data modeling skillsSnowflake and/or Databricks fundamentals, including performance and cost considerationsdbt framework design and transformation best practicesAirflow (or equivalent) orchestration and production support experienceAzure and/or AWS cloud-native data architectureAPI integration and service-oriented data patternsData quality, lineage, governance, and audit controlsCompetencies & Attributes Strong problem-solving and analytical skillsAbility to translate business workflows into scalable data solutionsClear communication with both technical and non-technical stakeholdersUnderstanding of BI/reporting architecture, semantic modeling, and analytics performance