Senior Software Engineer, Deep Learning Inference

Details

We are now looking for a Senior Software Engineer for Deep Learning Inference to make a big impact by helping build a state-of-the-art inference framework for accelerating Deep Learning models, especially Large Language Models (LLMs), on NVIDIA GPUs.

Responsibilities

• Develop components of TensorRT-LLM, NVIDIA’s best-in-class library for optimizing LLM inference performance on NVIDIA GPUs.
• Provide expert solutions to internal and external TensorRT-LLM users across GitHub and internal forums, and help manage TensorRT-LLM’s Open Source Software (OSS) repo on GitHub.
• Collaborate across diverse teams of deep learning experts, GPU architects, and DevOps engineers within NVIDIA, as well as the larger deep learning community, in an open-source development process.

Requirements

• A Bachelor's, Master's, PhD or equivalent experience in Computer Science, Computer Engineering, Electrical Engineering or related field.
• 6+ years of software development experience.
• Strong experience with Python.
• Strong grasp of Machine Learning concepts, especially related to Large Language Models.
• Excellent communication skills, and an aptitude for collaboration and teamwork.

Ways to Stand Out

• Strong experience with C++11/C++14/C++17.
• Background with OSS development (prior contributions to related deep learning projects a big plus!).
• Background in working with vLLM, TensorRT, PyTorch, JAX, or other ML frameworks.
• Experience collaborating with external customers and end users to resolve complex technical issues.
• Experience in software performance benchmarking, profiling, and optimizations.

Benefits

• Eligible for equity and benefits.
• NVIDIA is committed to fostering a diverse work environment and is an equal opportunity employer.

You will be joining a highly innovative and forward-thinking team working on critical technologies in deep learning.