- An agent platform - an extensible, Model Context Protocol (MCP)-based system of AI diagnostic skills integrated with enterprise data services, run inside engineers' existing tools (CLI, VS Code, and chat). - Production engineering rails - CI-enforced quality gates, evaluation regression testing, comprehensive data science telemetry, and supply-chain security that let us ship AI safely and at scale. Every system is instrumented to surface performance, adoption, and quality signals in real time. - Language-model pipelines - training, fine-tuning (including RLHF), and automated evaluation of models that reason over and assess support cases. - Quality-evaluation systems - rubric-based, LLM-as-judge evaluation that scores case handling and surfaces coaching and compliance signals. - Intelligent case-routing services - classification and ranking models that route cases to the right team and catch misrouted cases early. - Privacy-preserving data tooling - a redaction pipeline (pattern-matching → transformer models → LLM) that protects PII and secrets across every data flow, forming the trusted substrate the rest of the platform depends on. Bachelor's Degree in Computer Science, Information Technology (IT), or related field AND 3+ years technical support, technical consulting experience, or information technology experience OR equivalent experience. These requirements include, but are not limited to the following specialized security screenings: 3+ years of experience developing and shipping production software services using TypeScript/Node.js, Python, or C#/.NET. Experience building AI-powered applications, including large language models (LLMs), AI agents, prompt engineering, Model Context Protocol (MCP), machine learning models, or text classification/ranking systems. Experience designing cloud and data solutions using Azure, large-scale datasets, Azure Data Explorer (Kusto), and retrieval technologies such as keyword, vector, or semantic search. Experience evaluating and improving AI systems through model evaluation frameworks, RLHF, preference-based training, LLM-as-a-judge methodologies, labeled datasets, or benchmark development. Experience implementing software quality, security, and DevOps practices, including automated testing, code reviews, CI/CD pipelines, GitHub Actions, CodeQL, dependency management, secret scanning, and branch governance. Experience developing customer-facing or enterprise applications, including solutions for privacy, compliance, PII detection, data redaction, customer support, or operational workflows.
Want jobs like this matched to you?
SimpleCareer scores fresh postings against your résumé so you only see the matches that matter.