Senior System Software Engineer - Dynamo and Triton Inference Server

at Nvidia
📍 World
📍 Canada
📍 United States
USD 184,000-356,500 per year
SENIOR
✅ Remote

SCRAPED

Used Tools & Technologies

Not specified

Required Skills & Competences ?

Python @ 7 Algorithms @ 7 Distributed Systems @ 4 Hiring @ 4 gRPC @ 4 Protobuf @ 4 Rust @ 7 Debugging @ 7 HTTP @ 4 JSON @ 4 PyTorch @ 7 Agile @ 4 GPU @ 4

Details

We are hiring a Senior System Software Engineer to develop open-source, GPU-accelerated software for serving inference of trained AI models. The team builds back-end services and software to make design and deployment of AI models easier and accessible. You will work on Dynamo and Triton Inference Server and on software that supports distributed, high-performance inference workloads.

Responsibilities

  • Develop open source software to serve inference of trained AI models running on GPUs.
  • Build robust, scalable, high-performance software components for distributed inference workloads.
  • Balance priorities with team leads and prioritize features and capabilities.
  • Load-balance asynchronous requests across available resources and optimize prediction throughput under latency constraints.
  • Integrate and utilize the latest open source technology.

Requirements

  • Masters, PhD, or equivalent experience in Computer Science, Computer Engineering, or a related field.
  • 8+ years of relevant experience.
  • Strong programming and software design skills in Rust, Python, and C++ including debugging, performance analysis, and test design.
  • Experience with high-scale distributed systems and ML systems.
  • Ability to work in a fast-paced, agile team environment.

Ways to stand out

  • Prior work improving performance of AI inference systems.
  • Background with deep learning algorithms and frameworks, especially experience with Large Language Models and frameworks such as PyTorch, TensorRT, and ONNX Runtime.
  • Experience building and deploying cloud services using HTTP REST, gRPC, protobuf, JSON and related technologies.
  • Familiarity with the latest AI research and practical knowledge of efficient implementations of these systems.

Benefits

  • Eligible for equity and company benefits.
  • NVIDIA emphasizes a diverse work environment and is an equal opportunity employer.

Compensation & Additional Details

  • Base salary ranges provided by level: Level 4: 184,000 USD - 287,500 USD; Level 5: 224,000 USD - 356,500 USD. Actual base salary will be determined based on location, experience, and pay of employees in similar positions.
  • Applications accepted at least until July 29, 2025.