Systems Engineer, Inference

Airon.ai builds and operates AI infrastructure and dedicated bare-metal NVIDIA GPU compute for enterprise customers. The Systems Engineer, Inference will own the production serving path for open-weight large language models, from GPU kernels through gateways, while improving performance, scalability, reliability, observability, and cost efficiency.

Responsibilities:

• Own solutions end-to-end — from architecture and implementation to deployment, monitoring and continuous improvement
• Contribute to technical decisions and help shape our architecture, engineering principles and ways of working
• Improve the reliability, performance and scalability of our technology as the business grows
• Use AI and modern engineering tools to increase development speed, quality and operational efficiency
• Own the serving path for large language models in production, from GPU kernels up to the gateway that fronts them
• Profile inference workloads end to end: prefill vs decode behavior, KV cache growth, memory bandwidth limits, SM occupancy, and arithmetic intensity (FLOPs per byte)
• Write and optimize kernels, cut launch overhead with CUDA graphs, and prove the gains in Nsight
• Tune and extend serving engines, mostly vLLM: continuous batching, paged KV cache, scheduler and chunked-prefill policy, and the throughput/latency tradeoff behind them
• Cut memory and cost through prefix caching, KV quantization and offloading, and weight quantization, while measuring quality regression honestly against the speed gained
• Ship latency wins with speculative decoding and the right tensor/pipeline parallelism split for each model and cluster
• Scale inference across nodes: NVLink within the node, Spectrum-X Ethernet across them, disaggregated prefill/decode, MoE expert parallelism
• Keep it reliable: gateway routing and fallback chains, rate limits, TTFT and ITL SLOs, observability, chaos testing
• Run the economics: queue-based autoscaling, cold start mitigation, spot GPU strategy, cost per million tokens, and MFU
• Publish benchmarks and writeups that others can reproduce