Member of Technical Staff - Research

Vals AI develops AI evaluation technologies and benchmarks for foundation models and generative AI systems. The Member of Technical Staff - Research will design and build novel benchmarks, conduct validation research, analyze model performance, collaborate with model labs and enterprises, publish findings, and implement benchmark designs at scale.

Responsibilities:

• Design and develop novel, high-impact benchmarks that assess challenging real-world capabilities
• Conduct research to ensure our benchmarks are valid, reliable, and meaningful
• Collaborate with foundation model labs and enterprises to understand evolving evaluation needs
• Analyze model performance across benchmarks and articulate results
• Publish research findings and contribute to the broader evaluation research community
• Work closely with the infrastructure team to implement your benchmark designs at scale
• Stay current with the latest developments in LLM capabilities and evaluation methodologies