Senior Applied AI Engineer - Health Tech AI Startup

AgentGraph is running a confidential search for a healthcare technology company modernizing how clinical and administrative documents move between providers, payers, and patients. The Senior Applied AI Engineer will design, build, and operate LLM-powered services for document understanding, structured extraction, retrieval, and agentic workflows, while owning evaluation, observability, optimization, customer escalations, and technical mentorship.

Responsibilities:

• Design, develop, and maintain the LLM-powered services and pipelines at the core of the platform — document understanding, structured data extraction, classification, and summarization across highly variable real-world clinical and administrative documents
• Build agentic and multi-step workflows where they demonstrably outperform simpler approaches — routing, validation, exception handling, and human-in-the-loop review — with guardrails and fallbacks appropriate to a clinical setting
• Select the right approach for each problem — hosted frontier models, fine-tuned open-weight models, or classical ML — based on quality, latency, and cost, and implement fine-tuning where it demonstrably beats prompting and retrieval on those measures
• Create and maintain the high-quality datasets that model development, fine-tuning, retrieval, and evaluation depend on, including labeling strategy for documents where ground truth is genuinely ambiguous
• Partner with Product Management and Quality Assurance so AI solutions meet real customer needs and performance requirements, following the company's software development life cycle and its reliability, scalability, and reproducibility standards
• Build and maintain evaluation suites for LLM-powered features — curated golden datasets, automated scoring calibrated against human review, and regression detection — so that model, prompt, and pipeline changes ship with evidence rather than intuition
• Establish observability for production AI services, including request tracing, token and cost accounting, and monitoring for quality drift and failure modes such as malformed or unfounded model output. In healthcare, a confidently wrong extraction is the failure mode that matters most, and the systems you build should surface it rather than pass it downstream
• Lead code reviews with particular attention to ML- and LLM-specific concerns such as data processing, prompt and model versioning, experiment tracking, evaluation coverage, and production readiness
• Provide problem-solving support for ML and AI systems, including performance optimization, debugging model and retrieval behavior, and improving pipeline efficiency
• Optimize inference for production environments — latency, throughput, token and compute cost, batching, caching, model routing, and model right-sizing — treating inference cost as a product constraint, not an afterthought
• Optimize ML pipelines for scalability, latency, and resource utilization at real production document volumes
• Serve as a technical point of contact for escalated customer issues involving ML and AI services, partnering with Customer Success to own each issue end to end — reproduce, root-cause, fix, and close the loop with affected stakeholders
• Participate directly in customer-facing engagements when required — solution design sessions, technical discussions, and integration support — representing the team's capabilities credibly to technical and non-technical audiences, and translating customer needs into engineering direction
• Feed lessons from customer engagements back into product, dataset, and system improvements
• Drive complex projects from ambiguous problem statements through design, implementation, and measured rollout, actively identifying and pursuing high-impact opportunities that align with company strategy
• Influence the architecture of ML/LLM systems and mentor engineers by sharing best practices, guiding implementation efforts, and providing constructive feedback that raises the team's overall technical standard

Requirements:

• Substantial production experience building LLM-powered systems that real users depend on — not prototypes, demos, or notebooks that never met a difficult document
• Depth in document understanding and structured extraction, and a realistic view of what OCR, layout analysis, and scanned or faxed source material actually do to a pipeline
• An evidence-driven practice: you build the evaluation before you tune, and you can show how a change moved a measured outcome
• Judgment about approach — knowing when a fine-tune, a retrieval pipeline, or plain classical ML beats reaching for the largest available model
• Comfort owning production systems, including debugging under pressure and treating cost and latency as things you are accountable for
• The ability to handle regulated data responsibly. Experience working under HIPAA with PHI is a strong plus; a willingness to work carefully within those constraints is a requirement
• Enough customer-facing ability to be trusted in a solution design session with a customer's technical and clinical stakeholders
• Remote-first — open to candidates across the United States
• This role has the ability to be up to 25% customer-facing. The work is primarily engineering, but the successful candidate should be comfortable stepping into customer-facing solution design, technical discussions, and escalation support as needed
• Experience working under HIPAA with PHI is a strong plus