Generative AI Engineer

Magpie Health Analytics is seeking an experienced AI Engineer to develop secure and trustworthy generative AI solutions for healthcare clients. The role focuses on building natural-language search and decision-support capabilities for large bodies of program documentation, including document ingestion, retrieval, evaluation, guardrails, deployment, monitoring, and responsible AI oversight.

Responsibilities:

• Design, develop, and maintain generative AI solutions that ground large language model (LLM) responses in authoritative source documentation
• Build and maintain document ingestion pipelines, including parsing, chunking, metadata tagging, embedding generation, and vector/semantic search indexing
• Collaborate with cross-functional teams and subject matter experts to understand user needs and translate them into requirements, prompts, and evaluation criteria
• Implement responses with source citations and direct links to the underlying documentation to ensure traceability
• Design and implement guardrails, including content filtering, scope restrictions, PII/PHI protections, and refusal handling for out-of-scope questions
• Develop evaluation frameworks and test sets to measure retrieval quality, answer accuracy, groundedness, and consistency; monitor and tune performance over time
• Deploy and operate solutions within approved AWS services and federal security controls, supporting documentation needed for security authorization and AI governance reviews
• Implement logging, monitoring, and audit trails to support transparency, cost management, and responsible AI oversight
• Write clean, reusable, and well-documented code, and participate in code reviews, testing, and deployment processes
• Stay up to date with emerging AI technologies, models, and federal AI guidance