Senior Deep Learning Software Engineer, Inference and Model Optimization

Details

NVIDIA’s Algorithmic Model Optimization Team focuses on optimizing generative AI models, including large language models (LLMs) and diffusion models, for maximum inference efficiency. The team applies techniques such as neural architecture search, pruning, sparsity, quantization, and automated deployment strategies. Its work includes applied research and development of the TRT Model Optimizer software platform, which is used by NVIDIA and external research and engineering teams.

The Senior Deep Learning Software Engineer will develop and scale automated inference and deployment solutions. The role spans high-level machine learning frameworks such as PyTorch and Hugging Face, as well as high-performance kernel implementations using CUDA, TRT-LLM, and Triton.

Responsibilities

• Train, develop, and deploy generative AI models such as LLMs and diffusion models using NVIDIA’s AI software stack.
• Use and extend the Torch 2.0 ecosystem, including TorchDynamo, torch.export, and torch.compile, to analyze and extract standardized model graph representations from arbitrary Torch models.
• Develop high-performance inference optimization techniques, including automated model sharding, tensor parallelism, sequence parallelism, efficient attention kernels, and KV caching.
• Collaborate with teams across NVIDIA to incorporate performant kernel implementations into the automated deployment solution.
• Analyze and profile GPU kernel-level performance to identify hardware and software optimization opportunities.
• Improve inference performance across NVIDIA’s inference software solutions, including TensorRT, TRT-LLM, and TRT Model Optimizer.
• Architect and design a modular, scalable software platform that provides broad model support and optimization techniques.

Requirements

• Master’s degree, PhD, or equivalent experience in Computer Science, AI, Applied Mathematics, or a related field.
• At least 8 years of relevant work or research experience in deep learning.
• Excellent software design skills, including debugging, performance analysis, and test design.
• Strong proficiency in Python, PyTorch, and related machine learning tools such as Hugging Face.
• Strong algorithms and programming fundamentals.
• Good written and verbal communication skills.
• Ability to work independently and collaboratively in a fast-paced environment.

Preferred Qualifications

• Contributions to PyTorch, JAX, or other machine learning frameworks.
• Knowledge of GPU architecture and the compilation stack, with the ability to understand and debug end-to-end performance.
• Familiarity with NVIDIA deep learning SDKs such as TensorRT.
• Experience writing high-performance GPU kernels for machine learning workloads using technologies such as CUDA, CUTLASS, or Triton.

Benefits

• Competitive base salary ranging from USD 184,000 to USD 287,500 for Level 4, or USD 224,000 to USD 356,500 for Level 5, based on location, experience, and compensation for similar positions.
• Eligibility for equity and benefits.
• NVIDIA is an equal opportunity employer committed to an inclusive work environment.

Applications will be accepted at least until October 13, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes.