Senior Deep Learning Software Engineer, Inference

NVIDIA develops advanced GPU-accelerated computing technologies and software for sophisticated AI applications. The Senior Deep Learning Software Engineer will design, build, and optimize open-source inference frameworks and model-serving pipelines, improve performance for large language and generative AI models, and contribute to NVIDIA inference libraries and software solutions.

Responsibilities:

• Performance optimization, analysis, and tuning of DL models in various domains like LLM, Multimodal and Generative AI
• Scale performance of DL models across different architectures and types of NVIDIA accelerators
• Contribute features and code to NVIDIA’s inference libraries, vLLM and SGLang, FlashInfer and LLM software solutions
• Work with cross-collaborative teams across frameworks, NVIDIA libraries and inference optimization innovative solutions

Requirements:

• Masters or PhD or equivalent experience in relevant field (Computer Engineering, Computer Science, EECS, AI)
• 5+ years of relevant software development experience
• Excellent C/C++ programming and software design skills
• SW Agile skills are helpful
• Python experience is a plus
• Prior experience with training, deploying or optimizing the inference of DL models in production is a plus
• Prior background with performance modeling, profiling, debug, and code optimization or architectural knowledge of CPU and GPU is a plus
• GPU programming experience (CUDA, OAI TRITON or CUTLASS) is a plus
• Contribute to deep learning software projects, such as PyTorch, vLLM, and SGLang to drive advancements in the field
• Experience with Multi GPU Communications (NCCL, NVSHMEM)