Senior AI Inference Engineer

StackYak is building infrastructure that brings compute, GPU infrastructure, networking, and inference together. The Senior AI Inference Engineer will determine how models run on available hardware, operate production inference systems, and turn serving expertise into software and automated decisions.

Responsibilities:

• How models run. Placement, parallelism, precision, and memory strategy across single-GPU, multi-GPU, and multi-node — and the reasoning behind each choice, in a form someone other than you can audit
• The serving runtimes. vLLM, SGLang, TensorRT-LLM, and whatever displaces them: which one we use where, and what we do when the one we picked turns out to be wrong
• What we are willing to promise. Concurrency limits, latency and TTFT targets, throughput under load. You own the evidence behind a number before it becomes a commitment to a customer
• Benchmarking as an institution. Methodology, harness, reproducibility — and the standing to say that a result does not mean what someone wants it to mean
• Model lifecycle in production. Loading, startup, health, upgrades, capacity, and failure handling, all of it happening while customers are connected
• Multi-tenant versus dedicated serving, and where the line between them actually falls
• Inference incidents, including the ones where the serving runtime turns out to be innocent and the cause is GPU, driver, host, or network
• Turning all of the above into software, so that it stops being institutional memory with a single point of failure