AI Engineer

Charity Navigator is hiring an AI Engineer to build and ship production LLM-powered and agentic features that help donors search, compare, and understand nonprofit profiles. The role focuses on application-layer AI features, retrieval and data integrations, evaluation infrastructure, and monitoring and improving production systems in partnership with the engineering team.

Responsibilities:

• Build and ship production LLM-powered and agentic features – prompt orchestration, tool calling, embeddings, structured outputs – favoring reusable components over one-off solutions
• Build API Integrations, MCP servers, and retrieval-augmented generation (RAG) workflows that connect our existing data systems to vector search, so that AI features surface and explain our published ratings data accurately and cite the source of each figure
• Build and run evaluation infrastructure – prompt regression tests, LLM-as-a-judge evaluations, synthetic datasets – that catch feature quality regressions before users do
• Monitor AI features in production. Track accuracy, latency, token cost, and hallucination rates, then debug and tune prompts based on what you find
• Implement runtime safety and reliability guardrails that keep AI outputs within quality, compliance, and operational bounds
• Research and recommend orchestration frameworks, vector stores, and model providers, weighing cost and operational burden against speed. Final build-versus-buy calls are made jointly with the VP of Engineering
• Work directly with non-technical stakeholders to scope problems and give them honest assessments of what AI can and cannot do
• Evaluate emerging models and tools, use AI-assisted development extensively, and help raise team engineering standards through code review