Director of Product Management, AI Evals

Harness is the AI Software Delivery Platform company, led by technologist and entrepreneur Jyoti Bansal. They are seeking a Director of Product Management to lead the AI Evaluation module, focusing on how engineering teams measure, validate, and improve AI applications in production.

Responsibilities:

• Design AI-assisted experiences for eval authoring, failure analysis, and remediation recommendations
• Own the vision, strategy, roadmap, and P&L for Harness's AI Evals module
• Partner with engineering to deliver scalable, differentiated solutions that developers love
• Work closely with GTM teams (sales, marketing, customer success) on positioning, adoption, and revenue growth
• Engage directly with customers to shape the roadmap, improve adoption, and ensure measurable business impact
• Drive competitive positioning and track product metrics

Requirements:

• 8+ years of product management experience, including AI/ML technologies, developer tooling, or platform products
• Experience owning business outcomes including strategy, positioning, and revenue impact
• Strong communication and stakeholder management skills across product, engineering, and GTM teams
• Obsessed with customer success and outcome-driven
• Strong understanding of: LLM evaluation methodologies, CI/CD pipelines and developer workflows, AI observability, monitoring, and quality assurance, Agentic AI systems and the unique evaluation challenges they present
• Experience building AI/ML evaluation or observability platforms at scale
• Experience in building products from the ground up
• Familiarity with prompt engineering, fine-tuning pipelines, and dataset versioning
• Exposure to AI governance, responsible AI frameworks, or automated remediation
• Cloud-native and enterprise SaaS experience