Principal Software Engineer, GPU Firmware and GPU System Software — CSP Engagements
at Nvidia
USD 272,000-431,200 per year
Used Tools & Technologies
Not specified
Required Skills & Competences
Tag name is followed by "@" symbol and proficiency level value.
About proficiency levels:
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Security @ 4
GPU @ 4
Observability @ 4
AI @ 4
NVLink @ 4
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Details
We are seeking a Principal Software Engineer to join the CSP Engagements team as the technical focal point for GPU firmware and GPU system software. You will work directly with engineering teams of key cloud service provider (CSP) / hyperscale customers to ensure they can reliably manage, update, and operate NVIDIA GPU firmware at fleet scale. You will drive work streams with CSP engineering teams, incorporate customer feedback into NVIDIA's firmware and system software roadmap, and ensure customer-side automation and recovery procedures are ready before each firmware release.
Responsibilities
- Drive GPU firmware and system software work streams with CSP engineering teams — ensuring they understand GPU firmware architecture (VBIOS, InfoROM, microcontroller firmware), update sequencing, recovery procedures, and GPU power management.
- Gather and synthesize CSP feedback on firmware/software manageability, observability, security (e.g., multi-tenancy isolation, secure boot, attestation), and performance; champion those priorities into NVIDIA's GPU firmware/software feature roadmap and delivery plan.
- Drive GPU firmware update orchestration for large-scale deployments — multi-GPU update sequencing, rollback strategy, failure handling, and validation across hundreds of GPUs per rack.
- Serve as the technical focal point between NVIDIA and CSP firmware/software engineering — ensuring GPU behaviors (error recovery flows, thermal protection, power state transitions) are well-documented and accessible for customer integration.
- Identify cross-CSP GPU software/firmware issue patterns (common update failures, recovery gaps, configuration problems) and drive documentation, tooling, and test strategy improvements.
Requirements
- 15+ years of experience in GPU system software, GPU firmware, or accelerator platform engineering. BS or MS in Computer Science, Electrical Engineering, or related field (or equivalent experience).
- Deep understanding of GPU architecture internals: streaming multiprocessors, GEMM execution, compute kernels, memory hierarchy, and how firmware/driver decisions impact GPU compute performance.
- Understanding of multi-GPU fabric architectures (NVLink or similar) and how firmware coordinates across multiple GPUs in a rack-scale system.
- Understanding of GPU firmware architecture: VBIOS, GPU microcontroller firmware, InfoROM, and their interaction with the GPU driver stack.
- Experience with firmware update lifecycle management at scale: multi-device update sequencing, A/B updates, rollback, staged rollout, emergency recovery.
- Understanding of GPU error handling and recovery flows — how firmware-level errors propagate through the driver stack to application-visible failures.
- Experience with GPU health monitoring and telemetry: Xid errors, thermal events, power events, ECC counters, and their significance for firmware/software teams.
- Strong customer orientation: proven success influencing engineering teams to improve quality and fleet manageability.
Ways to Stand Out
- Direct experience with NVIDIA GPU VBIOS, GPU microcontroller firmware, or GPU driver internals.
- Background in GPU fleet management at 10K+ GPU scale — firmware rollout, health-based remediation, fleet-wide configuration management.
- Experience with GPU error taxonomy (Xid classification, NVLink error counters, ECC events) and building runbooks around GPU firmware behavior.
- Understanding of GPU security: secure boot chain, code signing, attestation, debug authentication, multi-tenancy isolation at the firmware level.
- Familiarity with GPU power management architecture and its impact on workload performance at fleet scale.
Compensation and Additional Info
- Base salary range: 272,000 USD - 431,250 USD (determined based on location, experience, and pay of employees in similar positions).
- You will also be eligible for equity and benefits.
- Applications for this job will be accepted at least until June 30, 2026.
- NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering an inclusive work environment and is an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.