Summary
✨ AI‑Generated
A Java and Spark developer is sought to design and build large-scale distributed data-processing applications. The role involves scalable batch and real-time pipelines, Spark optimization, data transformation and cleansing, and integration with cloud storage and modern data platforms.
Highlights
Onsite full-time role focused on large-scale distributed data processing. Offers hands-on work building scalable pipelines and big-data applications, optimizing Spark workloads, and working with major cloud and data-storage technologies.
Description
Position: Java Developer
Location: : Montreal Quebec , Canada (Onsite)
Job type: Fulltime
Job Summary
We are seeking a highly skilled Java Spark Developer with strong experience in designing and developing large-scale distributed data processing applications.
The ideal candidate will have expertise in Java, Apache Spark, Hadoop ecosystem, SQL, and Cloud technologies (AWS/Azure/GCP).
The candidate will be responsible for building scalable data pipelines, optimizing Spark applications, and collaborating with cross-functional teams to deliver high-performance data solutions.
Key Responsibilities
Design, develop, and maintain scalable big data applications using Java and Apache Spark.
Build and optimize batch and real-time data processing pipelines.
Develop Spark applications using Spark Core, Spark SQL, DataFrames, and Spark Streaming.
Work with large datasets stored in HDFS, Hive, S3, Delta Lake, or Snowflake.
Implement data transformation, cleansing, and aggregation processes.
Optimize Spark jobs for performance, memory utilization, and resource management.
Develop REST APIs and microservices using Spring Boot where required.
Collaborate with Data Engineers, Architects, Business Analysts, and DevOps teams.
Troubleshoot production issues and perform root cause analysis.
Implement CI/CD pipelines and automated deployment processes.
Ensure data quality, security, and compliance with enterprise standards.
Participate in code reviews and follow best coding practices.
Required Qualifications
Bachelor's degree in Computer Science, Information Technology, or related field.
5+ years of Java development experience.
3+ years of hands-on Apache Spark experience.
Strong proficiency in: Java 8/11/17 Apache Spark Spark SQL Hadoop Ecosystem (HDFS, Hive) SQL and Database Development Experience with: Spring Boot and Microservices Kafka or other messaging platforms RESTful APIs Git, Maven, Jenkins Linux/Unix environments Strong understanding of distributed computing concepts.
Experience tuning and optimizing Spark jobs.
Preferred Qualifications
Experience with cloud platforms: AWS (EMR, S3, Glue, Lambda) Azure Databricks Google Cloud Dataproc Experience with: Databricks Delta Lake Snowflake Airflow Kubernetes and Docker Exposure to Scala or Python.
Banking, Financial Services, Insurance, Retail, or E-commerce domain experience.
Technical Skills
Programming Languages
Java SQL Python (Preferred) Scala (Preferred) Big Data Technologies
Apache Spark Spark SQL Spark Streaming Hadoop Hive Cloud & Data Platforms
AWS EMR Databricks Snowflake Delta Lake DevOps & Tools
Git Jenkins Maven Docker Kubernetes Nice to Have
Experience with real-time streaming using Kafka and Spark Streaming.
Knowledge of Data Lake and Lakehouse architectures.
Experience implementing data governance and monitoring solutions.
Familiarity with Agile/Scrum methodologies.
