Principal Software Engineer, AI Inference Runtime

Arm is seeking a Principal Software Engineer to set the technical direction for its AI Inference Runtime team. The role focuses on designing distributed inference runtime capabilities, enabling and optimizing modern AI models, developing high-performance kernels and data-movement paths, and improving latency, throughput, reliability, and resource efficiency. The engineer will also partner with cross-functional teams, lead technical reviews, and mentor engineers.

Responsibilities:

• Define the architecture, interfaces, and roadmap for AI inference runtime capabilities, including abstractions that support evolving models, workloads, and compute platforms
• Enable new model architectures end to end through operator support, production validation, and optimization of scheduling, batching, model execution, memory management, and KV-cache efficiency
• Profile system bottlenecks and develop optimized kernels and data-movement paths across compute, memory, networking, and framework integration
• Evaluate new inference techniques and build benchmarking, regression, validation, and safe-rollout systems to improve latency, throughput, reliability, and resource efficiency
• Partner with cloud, framework, compiler, hardware, and research teams; lead technical reviews, mentor engineers, and establish meticulous performance-engineering practices