Talentpluto is a fast-growing, venture-backed AI infrastructure company specializing in reinforcement learning training data and evaluations. They are looking for a Research Engineer to enhance the quality assurance systems for training data, ensuring datasets are reliable and ready for evaluation.
Responsibilities:
- Define and enforce quality standards for training datasets used for RL training and evaluation
- Build tooling and workflows to audit supplier-generated datasets, including sampling strategies, validation pipelines (rule-based and model-assisted), and feedback loops
- Evaluate and implement human-in-the-loop review workflows where beneficial to improve quality and efficiency
- Partner with external data suppliers to debug quality issues, provide actionable feedback, and improve their data generation processes
- Integrate QA learnings into internal tools and supplier portals to reduce anomalies, inconsistencies, and edge cases over time
- Track QA outcomes and continuously improve processes, metrics, and documentation