Senior Software Engineer, AI Developer Experience

Vanta is hiring a Senior Software Engineer to build the infrastructure and developer workflows that make AI-powered engineering reliable and useful. The role focuses on background agents, agent evaluation systems, and tools that help engineers adopt and improve AI-enabled development practices.

Responsibilities:

• Plan, build, and improve background-agent infrastructure that enables reliable, trackable, and repeatable automated engineering work
• Develop systems for evaluating agent performance using task outcomes, quality measures, human intervention, cost, and other signals
• Create evaluation sets and feedback loops that help AI systems improve over time through regular review, tuning, and correction
• Improve how engineers discover, understand, and use shared AI tools and workflows
• Evaluate AI development tools, including code generation, AI-assisted review, and testing, against real engineering needs, and help integrate the ones that prove useful
• Develop visibility into which tools and workflows are useful, where they create friction, and how they should evolve
• Create clear guidance and documentation that helps engineers adopt new AI-enabled development practices
• Improve code review workflows by using feedback from real engineering work to identify and address recurring friction
• Influence the foundations of an autonomous software factory, including systems that help agents complete bounded engineering tasks safely and effectively
• Work directly with engineers and internal users to understand their needs, validate solutions, and make thoughtful product tradeoffs
• Help the team move ambiguous, dependency-heavy projects from early exploration through reliable delivery
• Raise the quality bar through thoughtful code reviews, technical design discussions, testing, observability, and attention to reliability and security
• Partner closely with engineers and cross-functional stakeholders while contributing as a dependable, collaborative teammate

Requirements:

• Substantial industry software engineering experience
• Experience taking technical leadership roles on teams or complex projects
• Experience improving AI systems, creating evaluation sets, and driving quality hill-climbing through measurement and iteration
• Strong software engineering fundamentals and experience designing, building, testing, and operating reliable systems
• Experience with developer tooling, AI-enabled workflows, agentic systems, infrastructure, or related technical domains
• A track record of working through ambiguity, decomposing complex problems, managing dependencies, and helping projects reach completion
• Experience working in rapidly scaling startups or larger companies, particularly in environments with a strong bias for action
• An action-oriented approach and a willingness to engage directly with users and contribute hands-on
• Product judgment and curiosity: you bring strong technical opinions while staying focused on building platforms that meet users where they are
• Strong written and verbal communication, with the ability to explain technical concepts clearly and create guidance that helps others adopt new tools
• A collaborative approach and a demonstrated ability to make meaningful contributions as part of a team
• Uses AI in day-to-day work to amplify their skills and strengthen their outcomes, applying it across routine workflows, actively building fluency through experimentation, and exercising sound judgment to ensure AI-assisted work meets a high quality bar