Software Engineer, Inference

Luma is hiring a Software Engineer, Inference to build and operate the systems that serve its models. The role focuses on integrating model architectures, scaling inference deployments, optimizing GPU workloads, and improving scheduling, reliability, and deployment tooling across clusters.

Responsibilities:

• Ship new model architectures by integrating them into the inference engine
• Collaborate across research, engineering, and infrastructure to optimize model efficiency and deployments
• Build internal tooling to measure, profile, and track the lifetime of inference jobs and workflows
• Automate, test, and maintain inference services for maximum uptime and reliability
• Manage and optimize inference workloads across clusters and hardware providers, and scale deployments across thousands of machines
• Build scheduling systems that use expensive GPU resources optimally while meeting SLOs, and maintain CI/CD for model checkpoints and SDKs

Requirements:

• Strong Python and system-architecture skills
• Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar
• Experience with queues, scheduling, traffic control, and fleet management at scale
• Experience with Linux, Docker, and Kubernetes, and with orchestration, deployment, and scheduling
• Familiarity with Redis and S3-compatible storage
• Modern networking stacks including RDMA (RoCE, InfiniBand, NVLink)
• High-performance large-scale ML systems (100+ GPUs)
• CUDA, and FFmpeg or multimedia processing