Key skills
Backend developmentProgramming languagesPlatform engineeringInfrastructureDeveloper toolingLLM-powered systemsRAG pipelinesAgentsEvaluation frameworksAI deployment cost modelingLatency managementSafety in AIAILLMRAGPrototypingCollaboration
About this role
Mercury is a company making a significant commitment to AI, aiming to transform isolated experiments into shared infrastructure for enhanced collaboration. The Senior Software Engineer - AI Engineering will help extend and scale the internal AI platform, enabling faster prototyping and iteration across the company while ensuring a strong knowledge layer for AI systems.
Responsibilities:
- Build and evolve MCP servers that connect internal systems and data sources into a coherent interface for agents and engineers
- Expand and operate our LLM gateway infrastructure: routing, rate limiting, cost attribution, and observability across teams
- Turn early patterns into durable defaults: shared prompt libraries, guardrails, and policy-as-code so teams can move fast safely
- Shape and maintain structured context artifacts—clean, reliable, agent-consumable—so LLMs working in Mercury's systems can reason accurately about our domain
- Improve internal knowledge discoverability and retrieval so both humans and agents can quickly find accurate answers
- Partner with domain teams to standardize key sources of truth, and keep them fresh
- Build and refine sandbox environments and tooling that let engineers experiment with AI safely and at speed
- Create self-service scaffolding so non-engineers—PMs, ops, finance—can prototype and deploy AI-powered workflows with minimal hand-holding
- Build playgrounds and evaluation harnesses so internal AI agents can be tested and iterated in controlled environments before hitting production
Requirements:
- Has 5+ years of backend development experience in complex, production systems—you've built things that other engineers depended on
- Is fluent across programming languages and can navigate platform engineering, infrastructure, and developer tooling without needing a map
- Has hands-on experience building LLM-powered systems—RAG pipelines, agents, eval frameworks—and has shipped at least one of these to production
- Understands the real tradeoffs in AI deployments: cost modeling, observability, latency, and safety—not just the exciting parts
- Is high-agency and self-directed. You can operate effectively without tightly-defined scope, find the highest-leverage work, and get it done
- Communicates clearly across technical and non-technical audiences—you can explain what you built and why it matters