Senior Systems Software Engineer, Workload Systems Engineering

Details

NVIDIA is seeking a senior systems software engineer to help engineering teams run demanding build, simulation, and validation workloads efficiently and reliably. The Workload Systems Engineering team connects customer workflows with the infrastructure that performs them. The team develops self-service mechanisms and clear diagnostics, investigates performance and reliability across system boundaries, and delivers measured infrastructure improvements with the teams that own the underlying components.

The role involves enabling application configuration, remote execution, workers, hosts, GPU and CPU resources, storage, data paths, and on-target systems. It requires direct system evaluation, including diagnosing slow or failing workloads, testing alternatives in representative scenarios, applying improvements collaboratively, and verifying customer benefits.

Responsibilities

• Build shared self-service and validation mechanisms that allow engineering teams to configure and use supported infrastructure workflows with less manual assistance.
• Provide customers and on-call teams with diagnostic views across application, submission, remote-execution, and host or target layers to identify the source of failures and slowdowns.
• Establish performance and reliability benchmarks for representative workloads.
• Use profiling, metrics, traces, and controlled experiments to identify impactful changes.
• Evaluate and improve compute and GPU selection, scheduling and placement, concurrency, worker and runner configuration, resource sharing, caching, and input/output paths.
• Make retry causes, recovery outcomes, and wasted execution capacity visible.
• Work with component owners to eliminate recurring failures and unnecessary re-execution.
• Implement and validate infrastructure-level changes with application, build and CI, remote-execution, and infrastructure teams.
• Measure effects on turnaround, successful efficiency, stability, usable capacity, and cost.
• Turn successful investigations into supported workflows, diagnostics, documentation, and clear customer handoffs.

Requirements

• More than 8 years of software engineering experience working with production compute infrastructure, distributed systems, large-scale build and test systems, or comparable systems software.
• Hands-on experience debugging complex performance or reliability issues across software and hardware boundaries using logs, metrics, traces, profiling, and experiments.
• Proficiency in a systems programming or automation language and experience delivering tested changes into production infrastructure.
• In-depth experience with Linux and proficiency in one or more of the following areas: GPU systems and drivers, scheduling and resource management, remote execution or CI infrastructure, storage and data paths, or host and container infrastructure.
• Ability to work across teams with different component ownership, turn ambiguous symptoms into a clear technical plan, and explain results to engineering customers.
• A degree in computer science, computer engineering, or a related field, or equivalent experience.

Preferred Qualifications

• Experience with GPU sharing and isolation, CUDA execution behavior, MIG, MPS, GPU driver diagnostics, or high-performance computing workloads.
• Experience with workload placement, CPU and GPU concurrency, NUMA, cloud-instance benchmarking, or cost and performance tradeoffs.
• Familiarity with Bazel, remote execution, build front ends, simulation or replay workloads, and hardware-in-the-loop validation.
• Experience improving file-system, caching, networking, or high-throughput data paths under concurrent workloads.
• A record of building customer self-service or observability tools and using incident and retry data to drive verified reliability improvements.

Benefits

NVIDIA offers competitive salaries, equity, and a comprehensive benefits package. NVIDIA is committed to fostering an inclusive work environment and is an equal opportunity employer.