Summary
✨ AI‑Generated
An enterprise technology team is seeking a Big Data Engineer to build and maintain data-processing solutions using Scala, Java, and Spark on a cloud platform. The role spans container orchestration, CI/CD, security, monitoring, infrastructure automation, API testing, and end-to-end quality assurance.
Highlights
Broad big-data engineering role covering cloud infrastructure, containerization, data processing, security, monitoring, infrastructure as code, and automated testing. Offers exposure to a modern enterprise data platform ecosystem.
Description
Tech skills: Scala, Java, Spark, Shell, Azure, Sonar, docker hub, container registry, nexus, Postman, Azure CLI
• Programming Language: Scala, Java, Spark
• Tools: Gitlab, Sonar, docker hub, container registry, nexus, Postman, Azure CLI
• Build Scripting: Gradle, Maven, Shell
• Packaging Tool/Technology: Shell
• Cloud Technology: Azure
• Hosting: Experience with container management platforms like Kubernetes and databricks
• Data: Storage accounts, PostgreSQL, knowledge on Spark and Data Bricks API,
• Security: Azure AD (SAML Token based, Single sign-on), Hashi corp Vault, Managed Identities (Service Principals)
• Monitoring: Log Analytics, Azure Metrics, Azure Monitor,
• Infrastructure as Code (IaC): ARM
Test Automation:
• Test Frameworks: Cucumber, Junit, Mockito
• Tools: Gitlab, Sonar, docker hub, container registry, nexus, Postman
• Build Scripting: Maven, Shell
• Service / API Test: I want to test your endpoints as thoroughly as possible
• End to End Test: of the integrated system (continuous integration can be a major help here)
• Solid knowledge of QA methodologies, test planning, system dependencies, and product integration phases
Key Responsibilities
1.
End-to-End Project Ownership
2.
AI/GenAI Solution Development
3.
Full Stack Development
4.
Architecture & Design Decision-Making
5.
Stakeholder Interaction
6.
Agile & Independent Working Model
Core Skills
Strong full stack development capabilitySolid understanding of system architecture and designExperience building end-to-end applicationsAI & Data Skills
Experience working with GenAI / LLM-based solutionsAbility to integrate AI into real-world use cases
