Senior Software Engineer - Automated Parallel Programming

at Nvidia

📍 Santa Clara, United States

$180,000-276,000 per year

SENIOR
✅ On-site

SCRAPED

Used Tools & Technologies

Not specified

Required Skills & Competences ?

Machine Learning @ 4 TensorFlow @ 4 Hiring @ 4 Communication @ 4 Parallel Programming @ 4 PyTorch @ 4

Details

The PyTorch Team at NVIDIA is hiring passionate parallel programmers. Join us to design and build the tools used by millions of AI practitioners deploying AI applications scalable to thousands of GPUs. Our team is responsible for the continual delivery of best-in-class experience on NVIDIA's hardware with PyTorch. Join our team and collaborate with many multi-disciplinary engineering teams within NVIDIA and internationally in the PyTorch open source community to deliver our customers the best of NVIDIA software.

Responsibilities

  • Crafting a code generation system to accelerate portions of a graph collected from a machine learning framework.
  • Partnering with NVIDIA’s hardware and software teams to improve GPU performance in PyTorch.
  • Design, build, and support production AI solutions used by enterprise customers and partners.
  • Optimize the performance of influential, modern Deep Learning models coming out of academic and industry research, for NVIDIA GPUs and systems.
  • Collaborating with internal applied researchers to improve their AI tools.
  • Advise design of new hardware generations.

Requirements

  • MS or PhD Computer Science, Computer Engineering, Electrical Engineering or a related field (or equivalent experience).
  • Parallel programming experience with writing optimized kernels in the NVIDIA CUDA Programming Language or similar parallel languages.
  • 4+ years of experience with C++ programming.
  • Demonstrated experience developing large software projects.
  • Excellent verbal and written communication skills.

Ways to stand out from the crowd

  • Proven technical foundation in CPU and GPU architectures, numeric libraries, modular software design.
  • A background in deep learning compilers or compiler infrastructure.
  • Expertise with optimized distributed parallelism techniques and it’s a bonus if that includes parallelizing Large Language Models!
  • Knowledge of heuristic generation that employs cost models, machine learning, or auto-tuning.
  • Contributions to PyTorch, Numpy, JAX, TensorFlow, OpenAI-Triton, Lightning Thunder, TVM, Halide or similar systems.

The base salary range is 180,000 USD - 276,000 USD. Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. You will also be eligible for equity and benefits. NVIDIA accepts applications on an ongoing basis.