AI Safety & Evaluation Engineer

Syren is focused on responsible AI deployment, and they are seeking an AI Safety & Evaluation Engineer to ensure their models are safe, accurate, and trustworthy in production. The role involves designing red-teaming tests, building guardrails, and developing evaluation frameworks to uphold AI standards.

Responsibilities:

• Design and run red-teaming and adversarial testing against models and agents (jailbreaks, prompt injection, misuse)
• Build guardrails: input/output filtering, policy enforcement, content moderation, and safe fallback behavior
• Develop hallucination detection and grounding checks; measure and reduce factual error rates
• Design evaluation frameworks and benchmarks for accuracy, safety, robustness, and bias — both offline and in production
• Define and enforce responsible AI standards, documentation, and deployment gates
• Partner with agent and ML teams to close safety gaps before and after release

Requirements:

• 3+ years in ML/AI, security, or evaluation-focused engineering
• Strong Python and deep familiarity with LLM behavior, failure modes, and prompt engineering
• Hands-on experience building evaluations, benchmarks, or guardrail/moderation systems
• Understanding of red-teaming, adversarial attacks, and hallucination mitigation techniques
• Rigorous, data-driven approach to measuring model quality and risk
• Experience with eval tooling and safety frameworks
• Background in AI security, alignment research, or trust & safety
• Familiarity with AI governance, regulatory, or compliance requirements