Technical Product Manager
We are looking for a highly technical, strategic, and outcome-oriented Staff Technical Product Manager to establish and scale AI Evaluation as a core engineering capability across the DT organization. The Eval Technical Product Manager will lead this transformation by making trusted Evals the evidence layer for AI. This is a dual-role position with responsibility for Platform Eval Strategy & Adoption and Enterprise Eval Innovation, including Custom Eval Innovation. Responsibilities include identifying gaps where out-of-the-box Evals don’t adequately measure enterprise AI quality, defining custom metrics and evaluation strategies, developing reusable Evals for domain-specific behaviors, and embedding Evals into the full AI lifecycle. Qualifications include 10+ years in Product Management roles focused on AI, and experience with LLMs, evaluation science, CI/CD, and AI platforms.
Job Details
Responsibilities
- Partner closely with Servicenow AI platform and product teams to understand, implement, validate, and drive adoption of out-of-the-box Evals, while influencing the platform roadmap based on enterprise needs.
- Identify gaps in platform evaluation capabilities and drive the definition and development of custom Evals, metrics, rubrics, datasets, and evaluation strategies.
- Establish evaluation approaches for AI agents, task completion, tool usage and multi-step workflows.
- Embed Evals into the full AI lifecycle: Define → Measure → Build → Evaluate → Improve → Release → Monitor.
Requirements
- Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving.
- 10+ years of experience in Product Management roles, with a focus on AI
- A technically strong Product Manager who combines AI/LLM expertise + Evaluation Science + Quality Engineering + Product Management + Platform Strategy
- Experience with LLM-as-a-Judge, rubrics, evaluation datasets, agent evaluation, AI observability, CI/CD and developer platforms
- ServiceNow AI Platform experience is a huge plus
Skills & Technologies
Related Opportunities
Discover more opportunities that match your interests and skills