In this role you will: Partner closely with data science, applied science, and engineering to own the data, models, agentic harness and measurement strategy for Copilot / VS Code experiences and connect them to real online outcomes. Define the online and offline metrics that tell us whether a feature is working and moving developer productivity. Perform deep quantitative and qualitative analysis of developer telemetry, usage funnels, and model-quality signals to surface unarticulated and unmet needs. Design, run, and interpret A/B flights and controlled experiments end to end, from hypothesis and power analysis through readout and ship/no-ship decisions. Drive prompt engineering and model-behavior improvements in partnership with applied scientists. Engage directly with the developer community - forums, GitHub, user studies, and early-adopter programs - to validate direction with real users. Analyze competition and market trends across AI coding assistants to build a coherent strategy for growing adoption and engagement. Define success metrics, communicate them clearly, measure them, and use them to iteratively improve the services. Relentless customer obsession paired with rigorous attention to data quality and detail. Comfort living in the data - you reach for a query or a notebook before an opinion. Active listening to understand (Growth mindset). Clarity in communication without ambiguity across all formats. Break down complex ideas and explain them simply with data. 6+ years of product management experience shipping enterprise or consumer products in the software industry. Experience building large-scale AI/ML products/services is preferred. Demonstrated expertise in data analysis and experimentation - designing and interpreting A/B flights, defining online and offline metrics, and driving decisions from telemetry. Hands-on comfort with data tools (notebooks, and dashboards such as Kusto/ADX, Power BI, or similar) to independently explore data and validate hypotheses. Familiarity with LLM and Copilot technologies, including prompt engineering, model evaluation, and awareness of the AI-assisted developer tooling landscape. Bachelor's degree or higher in Computer Science, Statistics, Data Science, or a similar discipline, or equivalent industry experience. An MBA is a bonus. Strong analytical skills and an aptitude for solving complex business problems.
Want jobs like this matched to you?
SimpleCareer scores fresh postings against your résumé so you only see the matches that matter.