AI Evals & Benchmarks: Building AI evals and benchmarks to ensure that the system stays healthy and does not regress across multiple scenarios. Throughput & Reliability: Scaling the pipeline by an order of magnitude and improve reliability. Compute, Capacity & Orchestration: Managing compute and AI infra at Windows-wide scale; smoothing capacity spikes across model-hosting backends; managing queuing/scheduling and pool prioritization infrastructure. Cost & Efficiency: Driving billing/consumption estimation and cost-of-goods modeling. Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python. These requirements include but are not limited to the following specialized security screenings: Master's Degree in Computer Science or related technical field AND 6+ years technical engineering experience. OR Bachelor's Degree AND 8+ years technical engineering experience. OR equivalent experience. Experience building or operating AI agent / LLM-based systems at production scale. Hands-on experience with Azure DevOps (ADO) pipelines and CI/CD automation. Demonstrated experience designing and operating large-scale distributed systems or cloud service backends (orchestration, scheduling, autoscaling, or multi-tenant compute).
Want jobs like this matched to you?
SimpleCareer scores fresh postings against your résumé so you only see the matches that matter.