Principal Security Research Manager

Principal Security Research Manager | Microsoft Careers Skip to main content Microsoft Careers Careers Careers Search Search jobs Sign in Single PositionView All JobsPrincipal Security Research ManagerUnited States, Washington, RedmondApply nowAdd to cartFind out how well you match with this jobUpload your resumeJob descriptionCompany and benefitsJob number200045023Date postedJul 24, 2026Work site3 days / week in-officeTravelLess than 25%ProfessionSecurity EngineeringDisciplineSecurity ResearchRole typePeople ManagerEmployment typeFull-TimeOverviewWe are building the evaluation backbone for safe, reliable, and efficient agentic engineering in Microsoft Security. This team will create systems that determine when an AI agent, model, prompt, tool, memory strategy, or orchestration pattern is ready to be used in production security and engineering workflows. The role is ideal for engineers who can operate end to end: understand the workflow, design the benchmark, build the harness, implement validators and graders, run experiments, analyze quality and cost tradeoffs, connect results to production feedback, and help teams make evidence-based release decisions. Why this role matters: Microsoft Security is moving toward agentic engineering systems for security triage, remediation, repo readiness, and scan-to-verified-closure workflows. Evals are the trust system for that shift. They help decide whether autonomy can safely expand, whether a release should stop, and which configuration achieves the required quality, safety, reliability, latency, and cost bar with the lowest practical human-review burden. Role mission As a Principal Security Research Manager on the AI Evaluation Systems team, you will build the common evaluation platform and methodology used by MSec agent programs. You will work across evaluation design, platform implementation, test infrastructure, telemetry, measurement, security workflow understanding, and production learning. Your work will make agentic systems measurable, reproducible, governable, and continuously improving. Responsibilities Design and build end-to-end evaluation harnesses for agentic security and engineering workflows, including triage, remediation, repo readiness, escalation, tool use, and scan-to-verified-closure paths. Create representative benchmark suites and golden datasets that include normal, edge, adversarial, failure-recovery, regression, and production-derived cases. Implement deterministic validators, automated graders, trace analyzers, result stores, comparison views, and workflow adapters that make evaluations repeatable and actionable. Measure task success, correctness, safety and policy compliance, failure recovery, latency, tool-call behavior, token usage, total cost per successful outcome, and human-review effort. Compare models, prompts, tools, memory strategies, policies, and orchestration patterns under consistent conditions and help teams understand quality-versus-efficiency tradeoffs. Integrate evaluations into engineering workflows, CI/CD, release gates, and decision processes so material agent changes are supported by reproducible evidence before production rollout. Connect offline evaluation results with production feedback, including accepted and rejected outputs, human overrides, incidents, rollbacks, escaped defects, and customer or service-health signals. Detect regressions, benchmark drift, evaluator miscalibration, and cases where eval scores improve while real-world outcomes do not. Partner with agent builders, product teams, security engineers, data scientists, program managers, and leadership to turn evaluation results into release recommendations and autonomy-boundary decisions. Convert learnings into reusable paved paths: playbooks, templates, onboarding guides, dashboards, scorecards, and reference implementations that can scale across MSec. What you will...