Principal AI Platform Engineer

PEMCO is a mutual insurance company. The Principal AI Platform Engineer builds and operates PEMCO’s AI platform end to end, including model infrastructure and serving, retrieval, production agents, observability, cost management, and governance.

Responsibilities:

• Own AI economics and observability. Build the consumption telemetry for the entire AI estate (cost per agent, per workflow, per business outcome; token usage; GPU utilization) and the LLM observability layer under it (LangFuse-class tracing, evaluation, drift monitoring). Leadership decisions about AI spend run on your data
• Run the AI gateway. A single gateway fronting every provider and model (LiteLLM-class or Azure APIM GenAI): routing, fallback chains, quotas, and per-agent cost capture. This is the enforcement point for the economics
• Stand up open-weight serving. High-throughput serving of open-weight models on Azure GPU capacity (vLLM, Triton/TensorRT-LLM), quantization, right-sizing. You build the sizing evidence that justifies or kills any future on-prem investment
• Build the retrieval layer. Vector stores, embedding pipelines, and document processing for in-house AI builds, on a governed platform rather than one-off deployments
• Operate production agents. Deployment, monitoring, incident response, and retirement for the agent fleet (MCP-based orchestration, per-agent least-privilege identity). When an agent supporting a business workflow fails, you own recovery
• Shape the architecture. Represent platform reliability, security, and economics in AI solution reviews across teams; constructively challenge designs with data and propose alternatives
• Hold the governance line. RBAC for models and agents, prompt/output guardrails, a seat on the AI Governance Working Group with authority to block deployments that do not meet the bar

Requirements:

• 6+ years in platform engineering, SRE, or technical operations at senior/lead scope
• 2+ years running LLM/AI systems in production (or 4+ years ML platform operations)
• Hands-on open-weight model serving (vLLM, Triton/TensorRT-LLM or equivalent), including quantization and GPU right-sizing
• LLM observability and evaluation experience (LangFuse, LangSmith, Arize class), or the demonstrated ability to stand it up
• Experience building RAG systems: vector stores, embedding pipelines, document processing
• FinOps/cost management for cloud consumption, ideally GPU or AI workloads
• Working knowledge of AI security risks: prompt injection, data leakage, model abuse
• on-prem GPU infrastructure design
• fine-tuning/LoRA
• agent frameworks (MCP, LangChain/LangGraph, Semantic Kernel)
• regulated-industry experience