Senior Software Engineer - LLM Inference

What We Do Products Solutions Support & Services Partners Resources Company Contact Us Global Offices Next Community Senior Software Engineer - LLM Inference Engineering Vancouver, British Columbia Req.Num.: 32047 Apply Hungry, Humble, Honest, with Heart. The OpportunityAt Nutanix, we're simplifying the future of AI with the Nutanix Cloud Platform for AI, enabling organizations to easily build, fine-tune, and run Generative AI, Large Language Models (LLMs), and next-generation Agentic AI applications without the complexity of managing AI infrastructure themselves. Our high-performance, full-stack machine learning cloud platform delivers AI-ready capabilities out of the box ("GPT-in-a-Box") through a software-defined, full-stack infrastructure solution that simplifies AI deployment across on-premises data centers, edge locations, and public clouds.The Enterprise AI team is at the forefront of this innovation, driving strategic products such as LLM Inference, the AI Gateway, and the Agentic AI Platform, recently showcased at NVIDIA GTC and NEXT 2026. Join the team responsible for the foundational AI technologies powering the next wave of intelligent applications at Nutanix. About the Team:We are a fast-paced, globally distributed team building the foundational layers of the enterprise AI stack. By joining our team, you'll help shape the next generation of enterprise AI platforms, working at the intersection of large-scale distributed systems and machine learning infrastructure. This is an opportunity to solve complex technical challenges, influence the direction of key AI technologies, and build systems that power AI workloads at enterprise scale. You will report to a seasoned Technical Manager who will provide mentorship and guidance as you navigate through your responsibilities. The work setup at Nutanix AI is a hybrid model, offering a blend of in-office collaboration and remote work flexibility. As a new hire, you will be expected to be in the office for 3 days a week, ensuring that you have the opportunity to engage with your team and foster strong working relationships.Your Role: Architect, design, and develop horizontally scalable, containerized, fault-tolerant services on Kubernetes.Improve the performance of systems to deliver for low-latency and high-throughput use cases.Optimize any part of the stack, including low-level systems.Leverage and contribute to relevant open-source cloud native projects.Develop scalable, efficient, and fault-tolerant observability architectures for collecting, analyzing, and reporting metrics for various platform services.Collaborate closely with globally located product management and backend development teams to deliver high-quality products in a fast-paced environment.Contribute to all stages of the product development cycle: technical design, development, test, experimentation, analysis, and launch.Be a team player by reviewing code and design docs, giving feedback on product specs and mocks, and documentation.Participate in an ongoing process definition and technology selection to ensure our technology stack is current with relevant trends.Continuously learn and improve your technical and non-technical abilities.What You Will Bring8+ years of experience developing maintainable, modular, resilient, fail-safe, and long-lasting code from a Product Development company.Have strong programming fundamentals, data structure, and algorithms.Strong experience in Docker, Kubernetes, and Cloud native technologiesExperience building applications with Go and PythonExperience building and managing CI/CD pipelinesStrong understanding of datacenter design, including computing, storage, and networking.Familiarity with on-prem, cloud, and hybrid software deployment architecturesGood experience in designing and tuning high-performance system software Strong understanding of distributed computing and storage...