Senior Software Engineer - NVLink Rack-Scale Stability and Reliability
at Nvidia
USD 152,000-287,500 per year
Used Tools & Technologies
Not specified
Required Skills & Competences
Tag name is followed by "@" symbol and proficiency level value.
About proficiency levels:
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Python @ 1
CI/CD @ 4
Distributed Systems @ 6
Hiring @ 4
Bash @ 1
Communication @ 7
Networking @ 4
SRE @ 4
Stress Testing @ 4
Debugging @ 7
System Architecture @ 7
CUDA @ 4
GPU @ 4
AI @ 4
InfiniBand @ 4
HPC @ 4
Performance Analysis @ 4
NVLink @ 4
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Details
NVIDIA is leading developments in Artificial Intelligence, High-Performance Computing and Visualization. The GPU, NVIDIA's invention, is at the heart of products and services that power modern AI and datacenter systems. The Fabric Networking team is hiring Senior Software Engineers focused on NVLink Rack-Scale Systems Stability & Reliability to help transform next-generation NVLink and NVSwitch platforms into stable, reliable, production-ready systems. The role addresses system-level resiliency, diagnostics, recovery, and large-scale AI infrastructure software.
Responsibilities
- Drive platform bringup, feature enablement, end-to-end software validation, and debug for next-generation NVLink-based GPU and rack-scale systems.
- Develop tools, diagnostics, automation, and infrastructure for system validation, regression testing, and fleet support.
- Lead reliability and MTBI validation through stress testing, telemetry analysis, failure injection, and issue resolution.
- Triage complex software, firmware, networking, and platform issues across validation, deployment, and production environments.
- Collaborate with architecture, hardware, firmware, software, and customer engagement teams to improve system quality and reliability.
- Build and maintain SRE-style validation infrastructure, including provisioning, monitoring, and operational readiness.
- Create automation, dashboards, runbooks, and debug workflows to improve root-cause analysis and operational efficiency.
Requirements
- BS or MS in Computer Science, Computer Engineering, Electrical Engineering, or related field, or equivalent experience.
- 5+ years of experience in system software, firmware, networking, platform enablement, data center infrastructure, or distributed systems.
- Strong programming skills in C/C++ and Python; Bash/Shell scripting experience is a plus.
- Strong system-level debugging across software, firmware, hardware, and networking layers.
- Solid networking fundamentals, including TCP/IP, Ethernet and/or InfiniBand, RDMA/RoCE, routing, switching, and fabric performance analysis.
- Experience with large-scale AI systems, including platform bringup, validation, reliability engineering, stress testing, telemetry analysis, and root-cause debugging.
- Ability to triage complex multi-domain issues using logs, telemetry, experiments, and structured debugging methods.
- Strong communication and collaboration skills across engineering, customer, and operations teams.
Ways to Stand Out
- Experience with NVIDIA GPU systems, NVLink, NVSwitch, CUDA, and large-scale AI/HPC clusters such as NVIDIA GB200 NVL72.
- Strong understanding of large-scale AI system architecture, including PCIe, memory hierarchy, DMA, high-speed interconnects, and distributed training/inference systems.
- Experience with server management technologies, data center operations, cluster provisioning, scaling, and fleet monitoring.
- Proven experience building diagnostics, automation, CI/CD pipelines, dashboards, and reliability tooling.
Compensation & Benefits
- Base salary range (dependent on level and location):
- Level 3: 152,000 USD - 241,500 USD
- Level 4: 184,000 USD - 287,500 USD
- Eligible for equity and company benefits (link to NVIDIA benefits).
Additional Information
- Applications accepted at least until May 26, 2026. This posting is for an existing vacancy.
- NVIDIA uses AI tools in recruiting processes.
- NVIDIA is an equal opportunity employer and committed to fostering a diverse work environment.