Director, Control Design – Evidence

Role Overview

• The Control Evidence & Assurance Lead is a senior technical leadership role responsible for ensuring AI controls are working through rigorous evaluation and documentation.
• You will design and assure the effectiveness of the evaluation framework that verifies the controls operate effectively
• Partner with control owners across the enterprise to maintain control efficacy
• Generate audit-ready evidence that proves controls reduce risk
• Translate control requirements into testable assertions
• Establish evidence traceability
• Ensure documentation meets regulatory and audit standards
• Partner closely with and inform the Governance Design Lead and Runtime Governance

Requirements

• Bachelor’s degree in Computer Science, Mathematics, Engineering, Risk Management, Systems Engineering, or related field with proficiency in evaluation frameworks and tools
• You have built and implemented evaluation frameworks that verify AI controls operate as intended and produce defensible, audit-ready evidence.
• You have demonstrated ability to review system architecture, evaluate technical implementations, and understand code well enough to validate that controls are effective.
• Hands-on coding experience (python) is a strong asset.
• You thrive in partnerships.
• Professional certifications in quality assurance, risk management, or auditing (e.g., FRM, CISM, CIA, or equivalent)
• Good understanding of LLM evaluation methodology including grounding, factuality, policy compliance, and adversarial testing
• Advanced degree in Computer Science, Mathematics, Engineering, Risk Management, or related discipline
• Direct experience with output assurance, content moderation, policy compliance evaluation at scale, or model risk management control design and verification
• Experience with adversarial attack methods, robustness testing, model monitoring systems, drift detection, or regulatory testing requirements (e.g., bias testing, fairness assessment)

Tech Stack

• Python

Benefits

• A comprehensive Total Rewards Program including bonuses
• flexible benefits
• competitive compensation
• stock where applicable
• Leaders who support your development through coaching and managing opportunities to build deep expertise in AI evaluation and control verification
• Flexible work/life balance options that support your professional and personal priorities
• Opportunities to take on challenging, technical work that shapes how AI controls are verified and how evidence is captured for governance
• Access to cutting-edge evaluation tools and methodologies with opportunities for continuous learning and professional growth