Human Data Operations Manager

Cartesia is pioneering AI that learns and interacts like humans, driven by a team of PhDs from the Stanford AI Lab. They are seeking a Human Data Operations Manager to design and operate a global evaluation workforce, impacting model quality and customer outcomes through effective workforce management and operational performance.

Responsibilities:

• Design and implement workforce structure across languages, skill tiers, and use cases, including evaluators, auditors, and leads for TTS products
• Build capacity models to support continuous eval pipelines and data production workflows
• Own relationships with vendors such as data annotation firms and contractor platforms, negotiating rate cards, SLAs, and throughput guarantees
• Decide on build, buy, or hybrid workforce models and continuously benchmark cost and performance across regions
• Design multi-layer QA systems spanning self-checks, peer review, audits, and gold tasks
• Define and track inter-rater reliability, error rates by category, and annotator-level performance distributions
• Build escalation and retraining workflows to maintain quality at scale
• Run day-to-day operations including task allocation, throughput tracking, and SLA adherence
• Build systems to reduce evaluator fatigue, rotate task types, and maintain consistency across large-scale evaluations
• Partner with tooling teams to improve evaluator UX and with data teams to ensure clean, structured outputs for model training