Senior Software Engineer, Machine Learning Inference

Details

At NVIDIA, the TensorRT team develops deep learning inference software for NVIDIA AI accelerators. This role focuses on designing and implementing inference software optimizations that power AI applications on NVIDIA GPUs across datacenters, workstations, and PCs.

Responsibilities

• Design, develop, and optimize NVIDIA TensorRT and TensorRT-LLM to improve inference applications.
• Develop software in C++, Python, and CUDA for efficient deployment of large language models and generative AI models.
• Collaborate with deep learning experts and GPU architects to influence hardware and software design for inference.

Requirements

• Bachelor's, master's, or PhD degree, or equivalent experience, in Computer Science, Computer Engineering, or a related field.
• At least 4 years of software development experience on a large codebase or project.
• Strong proficiency in C++ and proficiency in Rust or Python.
• Experience developing deep learning frameworks, compilers, or system software.
• Excellent problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment.
• Strong communication skills and the ability to articulate complex technical concepts.

Preferred Qualifications

• Experience developing inference backends and compilers for GPUs.
• Knowledge of machine learning techniques and GPU programming with CUDA or OpenCL.
• Experience with LLM inference frameworks such as TensorRT-LLM, vLLM, or SGLang.
• Experience with deep learning frameworks such as TensorRT, PyTorch, or JAX.
• Knowledge of close-to-metal performance analysis, optimization techniques, and tools.

Compensation and Benefits

• Base salary range of $152,000–$241,500 for Level 3.
• Base salary range of $184,000–$287,500 for Level 4.
• Eligibility for equity and benefits.
• Applications will be accepted at least until August 9, 2026.

NVIDIA is committed to fostering an inclusive work environment and is an equal opportunity employer.