Senior Deep Learning Software Engineer, Inference

at Nvidia

📍 United States

USD 152,000-287,500 per year

SENIOR

✅ Hybrid

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

AI @ 4 Agile @ 1 Algorithms CUDA @ 1 Debugging @ 1 Deep Learning @ 1 GPU @ 1 GenAI Generative AI LLM NCCL @ 4 Profiling @ 1 PyTorch @ 6 Python @ 1 SGLang @ 6 Software Development @ 6 vLLM @ 6

Details

NVIDIA is seeking a Senior Software Engineer specializing in deep learning inference. You will help design, build, and optimize GPU-accelerated software powering sophisticated AI applications, including high-performance open-source frameworks for large-scale model serving and inference. The role focuses on improving platforms that support the deployment and serving of language models.

You will work closely with the deep learning community to implement the latest algorithms for public release in inference frameworks and drive performance improvements for state-of-the-art large language models and generative AI models across NVIDIA accelerators, including data center GPUs and edge SoCs. Relevant open-source tools and plugins include CUTLASS, OAI Triton, NCCL, and CUDA kernels.

Responsibilities

Optimize, analyze, and tune deep learning models across large language model, multimodal, and generative AI domains.
Scale deep learning model performance across different architectures and types of NVIDIA accelerators.
Contribute features and code to NVIDIA inference libraries, vLLM, SGLang, FlashInfer, and other large language model software solutions.
Collaborate with teams across frameworks, NVIDIA libraries, and inference optimization initiatives.
Implement and optimize model-serving pipelines.

Requirements

Master's degree, PhD, or equivalent experience in Computer Engineering, Computer Science, EECS, AI, or a related field.
5+ years of relevant software development experience.
Excellent C/C++ programming and software design skills.
Software Agile skills are helpful; Python experience is a plus.
Experience training, deploying, or optimizing deep learning model inference in production is a plus.
Experience with performance modeling, profiling, debugging, code optimization, or CPU and GPU architecture is a plus.
GPU programming experience with CUDA, OAI Triton, or CUTLASS is a plus.
Contributions to deep learning software projects such as PyTorch, vLLM, or SGLang are valued.
Experience with multi-GPU communications technologies such as NCCL or NVSHMEM is valued.

Benefits

Base salary of $152,000–$241,500 for Level 3 or $184,000–$287,500 for Level 4, depending on location, experience, and compensation for similar positions.
Equity and benefits.
NVIDIA is an equal opportunity employer committed to an inclusive work environment.

Applications will be accepted at least until October 16, 2026.