Staff Application Security Engineer, Expert AI Reviewer (PhD)

Cobalt builds expert data and evaluation infrastructure for AI developers. The Staff Application Security Engineer will design realistic application security tasks, create vulnerable environments and reference solutions, evaluate frontier AI model attempts, and refine tasks to identify genuine security capability gaps.

Responsibilities:

• Design self-contained command line tasks drawn from real application security work, such as finding and patching injection flaws, fixing broken authentication and access control, auditing a codebase for unsafe deserialization or memory safety bugs, remediating vulnerable dependencies, and securing a build and deployment pipeline
• Build the task environment, including the vulnerable application or codebase, write a reference solution, and write automated tests that verify both that the flaw is fixed and that the application still works
• Run your task against a frontier AI model, review its attempt, and refine the task until the failure reflects a real gap in the model's capability rather than ambiguity or trick wording
• Work with reviewers to bring each task to acceptance

Requirements:

• A PhD in computer science, software security, or a closely related field
• At least 10 years of hands-on experience in application security, secure software engineering, or product security
• At least one publication, either academic (for example a peer-reviewed paper) or professional (for example a conference talk, a published vulnerability disclosure, or a widely used open-source tool)
• Strong code review skills in at least two widely used languages, such as Python, JavaScript, Java, Go, C, or C++
• Fluency in the Linux command line, shell scripting, Docker, and Git
• The ability to write precise specifications that another engineer could follow without asking questions