Senior Software Engineer - Stream Processing

Introduction At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world. Your Role And Responsibilities About the Role The Stream Processing & Analytics (SPA) team is building an elastic, reliable, durable, cost effective, and performant stream processing engine for Confluent Cloud based on Apache Flink®. Stream processing is at the heart of innovations that unlock the next generation of use cases for Confluent Cloud. As a Senior Software Developer on the FlinkSQL team, you will help build and evolve the SQL engine at the core of Confluent Cloud for Apache Flink® — the layer that makes stream processing feel as simple as using a database. You will work on the SQL planner and optimizer, SQL semantics and runtime operators, the catalog and metadata integration with Apache Kafka ® and Schema Registry, and the Table API, so that customers can express powerful streaming and batch workloads in SQL without worrying about the underlying infrastructure. You will collaborate closely with other teams (for example, the Flink runtime and control-plane teams), but your primary focus will be on the SQL layer itself: usability, performance, and operability of Flink SQL on Confluent Cloud. This role is based in Europe, where most of the FlinkSQL team is located. You will work daily with teammates across Europe and with partner teams in North America. You will have the opportunity to collaborate closely with the Apache Flink Open Source community. What You Will Do Implement and improve features in the Flink SQL engine: Contribute to the SQL parser, logical planner, and optimizer (based on Apache Calcite), including new SQL syntax, built-in functions, and planner rules. Improve the correctness and performance of streaming SQL operators (for example, joins, aggregations, and changelog processing) as seen from the SQL layer. Make Flink SQL a Great Experience On Confluent Cloud Work on the integration between SQL and the Confluent ecosystem: how tables map to Kafka topics, Schema Registry schemas, data types, and formats (Avro, Protobuf, JSON). Contribute to features across the SQL surface, such as user-defined functions (UDFs), the Table API, and standard metadata interfaces like INFORMATION_SCHEMA. Improve Reliability And Correctness Of The SQL Engine Deliver well-scoped improvements independently and with high quality, from clarifying requirements through implementation, testing, and rollout — with guidance from senior engineers on larger designs. Investigate and fix planner and semantic bugs, and extend our semantic test coverage so customer queries behave predictably. Strengthen Observability And Operational Excellence Add and refine metrics, structured logging, and dashboards that make SQL statements easier to understand and debug for customers and on-call engineers. Participate in the team's on-call rotation, supported by tested runbooks and processes, and contribute to incident follow-ups. Operate what you build in production. Preferred Education Master's Degree Required Technical And Professional Expertise Experience and fundamentals: 2-3+ years of industry experience building backend services, data infrastructure, or distributed systems. Strong computer science fundamentals, including data structures, algorithms, and a solid understanding of concurrency and distributed systems basics. Proficiency in Java (preferred for this role) or another JVM language (for example, Scala), with the ability to contribute to a large, existing codebase. SQL And Data Processing A solid understanding of SQL and relational concepts, and genuine interest in how query engines work (parsing, planning, optimization, execution). Familiarity with data processing systems such as databases, query engines, or stream processors — through work experience, open source, or advanced coursework/projects. Cloud And Operations Experience running or operating services on at least one major public cloud (AWS, GCP, or Azure); exposure to Kubernetes-based environments is a plus. Willingness to own your code in production, including metrics, dashboards, alerts, and on-call participation. Ways Of Working Ability to deliver well-scoped features end-to-end with high quality: clarify requirements, implement, test, and roll out safely, seeking input from senior engineers where needed. Clear, pragmatic communication and constructive participation in design discussions and code reviews. High bar for code quality and testing, and a willingness to iterate based on feedback. Preferred Technical And Professional Experience Experience with Apache Flink SQL or the Table API, or with another SQL engine or large-scale data processing system (for example, Spark SQL, Trino, or a database engine). Familiarity with query planners and optimizers, or compiler construction (Apache Calcite is a plus). Understanding of streaming semantics such as event time, watermarks, and CDC handling. Experience with Apache Kafka, Schema Registry, or serialization formats like Avro and Protobuf. Contributions to open-source projects in streaming, databases, or distributed systems, ideally including Flink or related components.