Staff Software Developer II - Confluent

Introduction

At IBM Software, we transform client challenges into solutions. Building the world’s leading AI-powered, cloud-native products that shape the future of business and society. Our legacy of innovation creates endless opportunities for IBMers to learn, grow, and make an impact on a global scale. Working in Software means joining a team fueled by curiosity and collaboration. You’ll work with diverse technologies, partners, and industries to design, develop, and deliver solutions that power digital transformation. With a culture that values innovation, growth, and continuous learning, IBM Software places you at the heart of IBM’s product and technology landscape. Here, you’ll have the tools and opportunities to advance your career while creating software that changes the world.

Your role and responsibilities

The Stream Processing & Analytics (SPA) team is building an elastic, reliable, durable, cost-effective, and performant stream processing engine based on Apache Flink for Confluent Cloud. Within SPA, the Flink Autopilot team owns the systems that make Flink a true cloud-native, "zero-knob" experience — autoscaling, resource management, and worry-free operations that deliver the right amount of stream processing at any given moment, so customers can focus on their use case instead of managing infrastructure.

As a senior engineer on Autopilot, you'll independently drive complex engineering projects end to end within our domain, become a go-to expert for one or more areas of the system, and help the team raise the quality and operational health of the platform.

What You Will Do

• Drive complex projects end to end: Independently take projects from design through production within the Autopilot domain — for example, autoscaling logic, resource management, or job/task manager resilience. Break large efforts into clear milestones and deliver them with high quality

• Master a domain: Develop deep expertise in one or more areas of Autopilot and become the person the team relies on for that area, representing your components in cross-functional design and architecture discussions

• Write scalable, performant systems: Design and build highly scalable, fault-tolerant systems, and articulate your thinking through clear design documents, one-pagers, and proposals

• Operational excellence and on-call: Improve the stability, performance, and automation of the systems you own. Participate in the on-call rotation, drive and mitigate incidents, and proactively address recurring operational toil

• Raise the bar through reviews and mentorship: Review PRs and designs with constructive feedback, coach junior engineers and interns, and contribute to the team's interview question pool and hiring

• Communicate clearly: Demonstrate strong, succinct written and verbal communication to align a small group on technical direction and keep stakeholders informed

Required technical and professional expertise

• Deep Expertise involving 6 - 8+ years of relevant experience in stream processing or large-scale distributed systems and some familiarity with autoscaling, resource management, or scheduling in distributed data systems

• Strong fundamentals in distributed systems or stream processing, with a track record of independently designing, building, and shipping complex systems end to end

• Proficiency in Java and/or Scala (the core languages of the Flink engine), with the ability to contribute production code from day one

• Hands-on experience building and operating mission-critical systems in a public cloud environment (AWS, GCP, or Azure)

• Ability to own a technical area, make sound trade-offs, and align a small group on direction.

Preferred technical and professional experience

• Experience with Kubernetes and operating distributed applications in production

• Open Source Engagement: Recognized, impactful technical contributions to open-source stream processing projects, particularly Apache Flink.

• Experience working with Go in a cloud control-plane context

IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.