Senior Software Engineer, Machine Learning Inference

at Nvidia
📍 Santa Clara, United States
USD 184,000-356,500 per year
SENIOR
✅ Hybrid

SCRAPED

Used Tools & Technologies

Not specified

Required Skills & Competences ?

Software Development @ 7 Python @ 4 Machine Learning @ 4 Communication @ 7 Rust @ 4 LLM @ 4 PyTorch @ 4 CUDA @ 4 GPU @ 6

Details

At NVIDIA, we're at the forefront of innovation, driving advancements in AI and machine learning to solve some of the world’s most challenging problems. We are seeking talented and motivated engineers to join the TensorRT team in developing industry-leading deep learning inference software for NVIDIA AI accelerators.

Responsibilities

  • Design, develop and optimize NVIDIA TensorRT and TensorRT-LLM to supercharge inference applications for datacenter, workstations, and PCs.
  • Develop software in C++, Python, and CUDA for seamless and efficient deployment of state-of-the-art LLMs and Generative AI models.
  • Collaborate with deep learning experts and GPU architects throughout the company to influence hardware and software design for inference.
  • Work on inference software optimizations and close-to-metal performance analysis to enable high-throughput, low-latency AI applications.

Requirements

  • BS, MS, PhD or equivalent experience in Computer Science, Computer Engineering or a related field.
  • 8+ years of software development experience on a large codebase or project.
  • Strong proficiency in C++ (required). Knowledge of Rust or Python is also accepted.
  • Experience in developing deep learning frameworks, compilers, or system software.
  • Knowledge of machine learning techniques and GPU programming (CUDA/OpenCL).
  • Excellent problem-solving skills and the ability to learn and work effectively in a fast-paced, collaborative environment.
  • Strong communication skills and the ability to articulate complex technical concepts.

Ways to stand out

  • Experience in developing inference backends and compilers for GPUs.
  • Experience with LLM inference frameworks like TensorRT-LLM, vLLM, SGLang.
  • Experience with deep learning frameworks such as TensorRT, PyTorch, JAX.
  • Knowledge of close-to-metal performance analysis, optimization techniques, and tools.

Compensation & Benefits

  • Base salary ranges by level: Level 4: 184,000 USD - 287,500 USD; Level 5: 224,000 USD - 356,500 USD. Final base salary will be determined based on location, experience, and internal pay equity.
  • Eligible for equity and additional benefits (see NVIDIA benefits page).

Additional Details

  • Employment type: Full time.
  • Office policy: Hybrid (#LI-Hybrid).
  • Application acceptance at least until October 3, 2025.
  • NVIDIA is an equal opportunity employer committed to diversity and inclusion.