Senior Machine Learning Test Engineer

Autodesk is a global company that focuses on design and engineering software. They are seeking a Senior Machine Learning Test Engineer to define quality standards for ML systems and collaborate across research and product teams to ensure reliable evaluation of ML models and data.

Responsibilities:

• Define ML quality strategy and acceptance criteria across data, model, and system levels
• Design and maintain model evaluation suites, metrics, and test datasets
• Evaluating CAD RL model outputs for geometric validity or policy stability
• Defining structured rubrics that translate qualitative findings into measurable evaluation gates
• Testing ML Models from product side
• API Testing
• Automate ML QA workflows using Python and CI/CD (e.g., GitHub Actions, Jenkins)
• Create and maintain test harnesses for ML services and APIs
• Mentor teams on ML QA best practices and consistent evaluation standards
• Build quality gates for training and deployment pipelines (e.g., regression checks, drift detection)
• Contribute to multi-team projects and codebases, ensuring code quality and consistency
• Participate in code reviews and provide constructive feedback to peers
• Document and present findings and ideas across the company

Requirements:

• Bachelor's degree in Computer Science, Engineering, or equivalent experience
• 7+ years of professional experience in software engineering or QA for ML/AI systems
• Strong programming skills in Python, with experience in test automation
• Familiarity with popular CAD environments tooling
• Proficient in Automation and UAT test suite/framework
• Experience designing QA frameworks or platforms used by multiple teams
• Excellent problem-solving skills and attention to detail
• Strong communication and collaboration skills
• Understanding of software architecture and design patterns
• Ability to work in an agile development environment
• Experience with data validation tooling (e.g., Great Expectations) or labeling workflows
• Familiarity with ML frameworks (e.g., PyTorch, TensorFlow)
• Experience with CI/CD tools and processes
• Experience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)
• Familiarity with MLOps practices (model monitoring, drift, deployment checks)
• Experience with ML evaluation methods, metrics, and benchmarking
• Passion for learning new technologies and improving existing systems
• Experience with cloud providers (e.g., AWS, Azure, Google Cloud Platform)
• Experience testing ML services in production environments
• Knowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases)