Machine Learning Researcher (PhD)

Cobalt is hiring Machine Learning Researchers for a contract project evaluating whether frontier AI models can perform research work in a command line environment. Researchers will design and validate challenging machine learning research tasks, assess model attempts, and work with reviewers to bring tasks to acceptance.

Responsibilities:

• Design self-contained command line tasks drawn from the practice of machine learning research, such as implementing a method from a paper’s description alone, locating the error in a flawed experimental setup, reproducing a reported result and explaining a discrepancy, implementing a custom loss function or optimizer correctly, and running a sound statistical comparison between two models
• Build the task environment, including code, data and dependencies, write a reference solution, and write automated tests that verify whether a solution is correct. Because results can vary between runs, tests need to be deterministic or use well-justified tolerances
• Run your task against a frontier AI model, review its attempt with the same rigor you would apply to a paper under peer review, and refine the task until the failure reflects a real gap in the model’s capability rather than ambiguity or trick wording
• Work with reviewers to bring each task to acceptance