Senior Data Engineer

Senior Data Engineer – Databricks & Quantexa Start Date: January 1, 2027 End Date: June 30, 2028 Work Arrangement: Remote with occasional in-office meetings Location: Toronto Position Overview We are looking for a Senior Data Engineer with strong experience in Databricks, Apache Spark, and enterprise data integration to support the implementation of Quantexa within an established data environment. The primary focus of this position will be integrating Quantexa with existing enterprise data platforms, preparing and onboarding data, and ensuring that the underlying data ecosystem can effectively support entity resolution, relationship analysis, and advanced analytics. This is primarily an integration and data engineering position, rather than a Quantexa product-development or customization role. The successful candidate will focus on connecting disparate data sources, developing reliable ingestion and transformation processes, and ensuring data is structured and optimized for downstream Quantexa processing. Ideal Candidate The ideal candidate is an experienced Data Engineer with hands-on Databricks and Quantexa experience who understands how to bring fragmented enterprise data together into a reliable analytical environment. This individual should be comfortable working across source systems, data pipelines, cloud platforms, and downstream analytical applications. Strong candidates will be able to assess complex data environments, develop scalable integration patterns, resolve data quality challenges, and ensure information is properly prepared for Quantexa processing. The emphasis of the role is on integration, data onboarding, pipeline engineering, data quality, and platform readiness, rather than developing or heavily customizing the Quantexa product itsel Key Responsibilities Data Integration & Platform Engineering Lead technical activities associated with incorporating Quantexa into an existing enterprise data ecosystem.Design, develop, and optimize data pipelines using Databricks, Apache Spark, and cloud-based technologies.Integrate information from multiple internal and external source systems.Build scalable ETL/ELT workflows to prepare data for downstream analytics and Quantexa processing.Consolidate fragmented datasets into consistent and usable data structures.Develop efficient data ingestion and transformation patterns capable of handling large data volumes.Work with architecture and platform teams to establish effective integration approaches.Data Preparation & Quality Analyze source data and determine the appropriate structures and transformations required for platform ingestion.Prepare structured and unstructured data for entity resolution and relationship analysis.Validate data accuracy, completeness, consistency, and lineage throughout the integration process.Identify data quality issues and work with relevant teams to resolve source-system or transformation problems.Apply appropriate data modeling techniques to support analytical and operational requirements.Optimize data processing and storage within Databricks environments.Implementation & Operational Support Participate in end-to-end testing and validation of integrated solutions.Troubleshoot data pipeline, integration, and performance issues.Support deployment activities and production readiness.Assist with monitoring and ongoing optimization of data workflows.Document technical architecture, integration approaches, data flows, and operational procedures.Collaborate with business, architecture, platform, and engineering teams throughout implementation. Required Qualifications 5+ years of professional Data Engineering experience.Strong hands-on experience with Databricks and Apache Spark.Previous experience implementing, integrating, or supporting Quantexa solutions.Experience working with large-scale enterprise datasets, including structured and unstructured information.Strong SQL capabilities.Solid understanding of data modeling principles.Proven experience designing and developing enterprise-grade ETL/ELT pipelines.Experience integrating data from multiple sources, including databases, APIs, flat files, and other enterprise systems.Experience with at least one major cloud platform such as Azure, AWS, or GCP.Understanding of data governance, data quality, metadata, and data lineage concepts.Strong troubleshooting and analytical skills. Preferred Experience Experience in one or more of the following areas would be considered an asset: Financial services or bankingFinancial crime technologyAML / KYCFraud analyticsRisk managementCustomer intelligenceEntity resolutionNetwork or relationship analysisData enrichmentGraph-based analytics Additional technical experience with the following is also beneficial: Delta LakeLakehouse architecturesAzure Data FactoryPython / PySparkCI/CD pipelinesDataOps practicesQuantexa data structures and deployment architecture Note: We use AI tools to: obtain basic information, detect plagiarism, false employment history or references, categorize your skills, and do an initial match with job posting.