Principal Engineer Team Lead, Behavior Safety & Metrics

Motional is hiring a Principal Engineer Team Lead to lead evaluation initiatives across its autonomous-driving Prediction and Planning teams. The role defines evaluation strategies, metrics, data-mining approaches, and scalable tools to assess system behavior, dataset coverage, safety, comfort, and driving performance.

Responsibilities:

• Act as the central authority on evaluation for our Behaviors teams, bridging the gap between model development and system-level autonomous behavior
• Drive the evaluation strategy for our prediction and planning models, including open-loop and simulation-based assessments
• Design and implement advanced metrics to measure model impact on overall system behavior, leveraging ML-based approaches for robust, end-to-end model evaluation
• Incorporate concepts such as counterfactual reasoning and reaction time analysis into our evaluation frameworks to better understand system behavior
• Develop the strategy for data mining and assess the coverage of our training and evaluation datasets across Prediction and Planning
• Build necessary tooling and pipelines, identify gaps in our processes, and collaborate with machine learning engineers and roboticists to solve open-ended problems
• Cross-functional collaboration: Partner with Safety, Infrastructure, Operations, Legal, and the other Systems teams to drive design improvements and ensure features generate a net positive effect on system safety, comfort, and assertiveness and meet our launch criteria

Requirements:

• Bachelor's degree and 7+ years of experience in Computer Science, Systems Engineering, Robotics, or a similar technical field (or Master's/PhD and 5+ years of experience)
• Strong foundation in probability/statistics, data science, and scientific rigor applied to complex, multi-agent systems
• Proficiency in Python, SQL, and data analysis (visualization, statistics) in production environments
• Experience defining performance targets and acceptance criteria spanning safety, comfort, and assertiveness, producing the analysis and evidence to justify them
• Demonstrated ability to tackle technical challenges, analyze cross-disciplinary problems, and propose innovative, maintainable solutions
• Proven ability to lead initiatives, drive alignment, and ensure stakeholder engagement throughout the development process
• Working expertise in hazard and safety analysis methods such as STPA, HAZOP, FMEA, and hazard analysis and risk assessment
• Experience using statistical and ML methods to quantify performance, detect regressions, and reveal patterns of system behavior at scale
• Deep experience with Prediction and/or Planning for autonomous robotics, including evaluating ML-based systems in production environments
• Experience with public AV datasets and benchmarks (e.g., Waymo Open Dataset, nuScenes, etc.) and a deep understanding of standard industry metrics for trajectory forecasting and motion planning
• Prior experience using or building data pipelines at scale
• Excellent communication skills, capable of articulating complex concepts to technical peers and executive leadership alike, with a proven ability to collaborate across distributed teams
• Familiarity with SOTIF, ISO 26262, UL4600, and AVSC behavior competencies and other industry standards