Senior Software Engineer, Deep Learning Inference Workflows

at Nvidia
📍 Santa Clara, United States
USD 184,000-287,500 per year
SENIOR
✅ Hybrid

SCRAPED

Used Tools & Technologies

Not specified

Required Skills & Competences ?

Software Development @ 7 DevOps @ 4 Python @ 4 C @ 4 C++ @ 7 Machine Learning @ 7 Communication @ 4 Performance Optimization @ 4 NLP @ 7 LLM @ 4 PyTorch @ 4 GPU @ 4

Details

We are looking for a Senior Software Engineer to help build a state-of-the-art inference framework for accelerating deep learning models, especially Large Language Models, on NVIDIA GPUs. The role is on the TensorRT Workflows team and involves developing high-performance inference components, collaborating with deep learning experts, GPU architects and DevOps engineers, and optimizing deployment pipelines for trained models.

Responsibilities

  • Develop components of TensorRT, NVIDIA’s SDK for high-performance deep learning inference.
  • Use C++ and Python to build graph parsers, optimizers, and tools for effective deployment of trained deep learning models.
  • Build and improve tooling for inference workflows, model deployment, and performance optimization.
  • Collaborate with teams of deep learning experts, GPU architects, and DevOps engineers across diverse teams.

Requirements

  • Bachelor’s, Master’s, PhD or equivalent experience in Computer Science, Computer Engineering, Electrical Engineering or related field.
  • 6+ years of software development experience.
  • Strong experience with C++ (C++11/C++14/C++17).
  • Strong grasp of machine learning concepts, especially Natural Language Processing (NLP).
  • Excellent communication skills and aptitude for collaboration and teamwork.

Ways to stand out

  • Proficiency in Python.
  • Experience in software performance benchmarking, profiling, and optimizations.
  • Background in compiler development.
  • Experience working with TensorRT, PyTorch, ONNX Runtime, JAX, TRT-LLM, vLLM, SGLang or other ML frameworks.
  • Experience with HuggingFace Diffusers and Transformers libraries.

Compensation & Benefits

  • Base salary range: 184,000 USD - 287,500 USD (determined based on location, experience, and pay of employees in similar positions).
  • Eligible for equity and company benefits. See NVIDIA benefits for details.
  • Applications for this job will be accepted at least until July 29, 2025.

Additional information

  • The role is part of NVIDIA’s TensorRT Workflows team. #LI-Hybrid
  • NVIDIA is an equal opportunity employer and is committed to fostering a diverse work environment.