Senior System Software Engineer - Dynamo-Triton Inference Server

at Nvidia

USD 152,000-287,500 per year

SENIOR

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

AI @ 4 Agile @ 7 Communication @ 7 Debugging @ 7 Deep Learning @ 6 Distributed Systems @ 4 GPU @ 4 GitHub @ 4 LLM @ 4 Machine Learning @ 4 Networking @ 4 Performance Analysis @ 7 PyTorch @ 4 Python @ 3 Rust @ 6 TensorRT @ 4 vLLM @ 4

Details

NVIDIA is looking for a Senior System Software Engineer to work on the Dynamo-Triton Inference Server. The role is part of NVIDIA’s GPU-accelerated deep learning software team, building a high-performance AI inference platform that makes the design and deployment of AI models easier and more accessible.

Responsibilities

Develop GPU-accelerated AI inference serving software.
Contribute to feature development and drive broad customer adoption.
Drive the convergence of the Triton Inference Server and NVIDIA Dynamo stacks to establish a unified, high-performance inference platform.
Ensure feature parity while serving both large language model (LLM) and non-LLM workloads.
Participate actively in the open-source deep learning software engineering community.
Build robust software for production server and cloud environments.
Optimize and balance prediction throughput and latency.
Develop and adopt next-generation inference technologies.

Requirements

MS or PhD in Computer Science or a relevant field, or equivalent experience.
5+ years of professional experience working on deep learning software.
Excellent Rust and C++ skills.
Familiarity with Python.
Strong programming and software design skills, including debugging, performance analysis, and test design.
Experience with high-scale distributed systems and machine learning systems.
Strong communication skills and the ability to work in a fast-paced, agile team environment.

Preferred Qualifications

Experience with AI frameworks and engines such as TensorRT, PyTorch, ONNX, OpenVINO, vLLM, or TRT-LLM.
Knowledge of GPU memory management, cache management, or high-performance networking.
Experience with distributed systems programming.
Experience contributing to large open-source projects, including GitHub workflows, bug tracking, branching and merging code, open-source licensing issues, and handling patches.

Compensation and Benefits

The base salary is determined based on location, experience, and the pay of employees in similar positions. The base salary range is USD 152,000–241,500 for Level 3 and USD 184,000–287,500 for Level 4. Employees are also eligible for equity and benefits.

Applications will be accepted at least until July 6, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes and is an equal opportunity employer.