Founding Inference Infrastructure Engineer

The client is an AI infrastructure company building a cloud platform for specialised inference accelerators. The Founding Inference Infrastructure Engineer will build and own the production LLM inference serving stack, including request scheduling, batching, KV-cache management, autoscaling, reliability, and on-call operations while shaping the serving architecture and roadmap.

Responsibilities:

• Own the inference serving stack end-to-end. Design and build request routing, batching, scheduling and autoscaling across model-serving replicas
• Reduce cost per token. Optimise batching strategies, KV-cache management and hardware utilisation to improve throughput and efficiency
• Build reliability from day one. Establish monitoring, alerting and failover, and respond to incidents affecting customer workloads
• Partner with compiler and model bring-up engineers. Define the interface between a model that is correctly compiled and validated and one that is serving live requests efficiently
• Shape the serving roadmap. Help prioritise capabilities such as multi-tenant isolation, speculative decoding and new scheduling strategies
• Set the technical standard. Make architecture and code-quality decisions that will guide the serving team as it grows