Senior Deep Learning Software Engineer, Inference

at Nvidia

📍 World
📍 Canada
📍 United States

USD 152,000-287,500 per year

SENIOR

✅ Remote

Used Tools & Technologies

GenAI

Required Skills & Competences
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

Software Development @ 6 Python @ 1 Algorithms @ 4 Performance Optimization @ 4 Debugging @ 4 LLM @ 4 PyTorch @ 4 Agile @ 4 CUDA @ 4 GPU @ 4 Deep Learning @ 4 Generative AI @ 4 AI @ 4 Profiling @ 4 vLLM @ 4 NCCL @ 4 SGLang @ 4

Details

NVIDIA seeks a Senior Software Engineer specializing in Deep Learning inference. You will design, build, and optimize GPU-accelerated software that powers advanced AI applications. The team develops and maintains high-performance open-source frameworks for efficient large-scale model serving and inference, improving platforms for deployment and serving of large language models (LLMs) and generative AI across NVIDIA accelerators (datacenter GPUs to edge SoCs). The role involves working with the deep learning community to implement latest algorithms for public release and using open-source tools and plugins including CUTLASS, OAI Triton, NCCL, and CUDA kernels to implement and optimize model serving pipelines.

Responsibilities

Performance optimization, analysis, and tuning of deep learning models (LLM, multimodal, generative AI).
Scale performance of DL models across different architectures and NVIDIA accelerators.
Contribute features and code to NVIDIA inference libraries and projects such as vLLM, SGLang, FlashInfer and other LLM software solutions.
Collaborate across teams working on frameworks, NVIDIA libraries, and inference optimization solutions.

Requirements

Masters or PhD in a relevant field (Computer Engineering, Computer Science, EECS, AI) or equivalent experience.
5+ years of relevant software development experience.
Excellent C/C++ programming and software design skills; agile software development experience is helpful.
Python experience is a plus.
Prior experience with training, deploying, or optimizing DL model inference in production is a plus.
Background in performance modeling, profiling, debugging, code optimization, or architectural knowledge of CPU and GPU is a plus.
GPU programming experience (CUDA, OAI Triton, CUTLASS) is a plus.

Ways to Stand Out

Contributions to deep learning software projects (PyTorch, vLLM, SGLang).
Experience with multi-GPU communications (NCCL, NVSHMEM).

Benefits and Compensation

Base salary range (by level):
- Level 3: 152,000 USD - 241,500 USD
- Level 4: 184,000 USD - 287,500 USD
Eligibility for equity and benefits (link provided in original posting).

Additional Information

Applications accepted at least until July 17, 2026.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is an equal opportunity employer and emphasizes an inclusive work environment.