Senior Software Engineer - NIM Factory Container and Cloud Infrastructure

at Nvidia

USD 184,000-356,500 per year

SENIOR

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

API @ 4 CI/CD CUDA @ 4 Communication @ 6 Docker @ 4 GPU @ 4 Helm @ 6 Kubernetes @ 7 LLM @ 4 Microservices Networking Python @ 7 SGLang @ 4 SRE Security vLLM @ 4

Details

NVIDIA is seeking a Senior Software Engineer focused on container and cloud infrastructure. The role involves designing and implementing the core container strategy for NVIDIA Inference Microservices (NIMs) and hosted services, building enterprise-grade software and tooling for container build, packaging, and deployment, and improving reliability, performance, and scale across thousands of GPUs. The team will also support disaggregated LLM inference and other emerging deployment patterns.

Responsibilities

Design, build, and harden containers for NIM runtimes and inference backends, enabling reproducible, multi-architecture, CUDA-optimized builds.
Develop Python tooling and services for build orchestration, CI/CD integrations, Helm and Operator automation, and test harnesses.
Enforce quality through typing, linting, and unit and integration testing.
Help design and evolve Kubernetes deployment patterns for NIMs, including GPU scheduling, autoscaling, and multi-cluster rollouts.
Optimize container performance, including layer layout, startup time, build caching, runtime memory and I/O, networking, and GPU utilization.
Instrument systems with metrics and tracing.
Evolve base image strategy, dependency management, and artifact and registry topology.
Collaborate with research, backend, SRE, and product teams to ensure day-0 availability of new models.
Mentor teammates and establish high engineering standards for container quality, security, and operability.

Requirements

BS or MS degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
6+ years of experience building production software, with a strong focus on containers and Kubernetes.
Strong Python skills for building production-grade tooling and services.
Experience with Python SDKs and clients for Kubernetes and cloud services.
Expert knowledge of Docker/BuildKit, containerd/OCI, image layering, multi-stage builds, and registry workflows.
Deep experience operating workloads on Kubernetes.
Hands-on experience building and running GPU workloads in Kubernetes, including the NVIDIA device plugin, MIG, CUDA drivers and runtime, and resource isolation.
Excellent collaboration and communication skills, with the ability to influence cross-functional design.

Additional Qualifications

Expertise with Helm chart design systems, Operators, and platform APIs serving many teams.
Experience with the OpenAI API and Hugging Face API, as well as an understanding of inference backends such as vLLM, SGLang, and TRT-LLM.
Background in benchmarking and optimizing inference container performance and startup latency at scale.
Experience designing multi-tenant, multi-cluster, or edge and air-gapped container delivery.
Contributions to open-source container, Kubernetes, or GPU ecosystems.

Benefits

The position offers competitive salaries, a generous benefits package, and equity. NVIDIA is an equal opportunity employer committed to fostering a diverse work environment. Applications will be accepted at least until April 13, 2026.