Principal AI and ML Infra Software Engineer, GPU Clusters

at Nvidia

USD 272,000-431,200 per year

SENIOR

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

AI @ 4 AWS @ 3 Azure @ 3 Bash @ 6 Cloud Computing @ 3 Communication @ 6 DevOps Docker @ 4 GCP @ 3 GPU Go @ 6 HPC @ 4 InfiniBand @ 4 JAX @ 4 Kubernetes @ 4 Machine Learning Networking @ 4 PyTorch @ 4 Python @ 6 Slurm @ 4

Details

NVIDIA is seeking a Principal AI and ML Infra Software Engineer, GPU Clusters to join its Hardware Infrastructure team. This role focuses on improving efficiency for AI and ML researchers by implementing enhancements throughout the infrastructure stack. You will collaborate closely with customers and research teams to identify and address infrastructure deficiencies, enabling groundbreaking AI and ML research on GPU clusters.

Responsibilities

Engage closely with AI and ML research teams to understand their infrastructure requirements and challenges, translating those insights into actionable improvements.
Identify researcher-efficiency bottlenecks and lead initiatives to systematically improve efficiency.
Drive the direction and long-term roadmaps for infrastructure-efficiency initiatives.
Monitor and optimize infrastructure performance to ensure high availability, scalability, and efficient resource utilization.
Help define and improve key measures of AI researcher efficiency, ensuring that initiatives produce measurable results.
Collaborate with researchers, data engineers, DevOps professionals, and other teams to develop a cohesive AI/ML infrastructure ecosystem.
Stay current with developments in AI/ML technologies, frameworks, and best practices, and advocate for their integration within the organization.

Requirements

Bachelor's degree or equivalent background in Computer Science or a related field, or equivalent experience.
15+ years of demonstrated expertise in AI/ML and high-performance computing tasks and systems.
Hands-on experience using or operating HPC-grade infrastructure.
In-depth knowledge of accelerated computing, including GPUs and custom silicon; storage technologies such as Lustre, GPFS, and BeeGFS; scheduling and orchestration technologies such as Slurm, Kubernetes, and LSF; high-speed networking such as InfiniBand, RoCE, and Amazon EFA; and container technologies including Docker and Enroot.
Experience supervising and improving large-scale distributed training operations using PyTorch, including DDP and FSDP, NeMo, or JAX.
In-depth understanding of AI/ML workflows, including data processing, model training, and inference pipelines.
Proficiency in programming and scripting languages such as Python, Go, and Bash.
Familiarity with cloud computing platforms including AWS, GCP, and Azure.
Experience with parallel computing frameworks and paradigms.
Commitment to ongoing learning and staying current with new technologies and methods in AI/ML infrastructure.
Excellent communication and collaboration skills, with the ability to work effectively with teams and individuals from diverse backgrounds.

Compensation and Benefits

The base salary range is $272,000–$431,250 USD, determined by location, experience, and compensation for similar positions. The role is also eligible for equity and benefits.

Applications will be accepted at least until October 18, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes and is an equal opportunity employer.