Senior Software Engineer, Distributed Systems - NIM Factory

at Nvidia

USD 184,000-287,500 per year

SENIOR

✅ On-site

Tech Stack
Tag name is followed by "@" symbol and proficiency level value. About proficiency levels:

1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;

3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;

7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;

10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.

AI @ 4 CI/CD @ 4 Docker @ 6 GPU Helm @ 6 Kafka @ 4 Kubernetes @ 6 Mentoring @ 4 Microservices @ 7 Observability Prometheus @ 6 Redis @ 4

Details

NVIDIA is seeking a senior engineer to design and build factory infrastructure and automation for NVIDIA Inference Microservices (NIMs). You will help develop infrastructure that optimizes and serves performant inference for AI models across heterogeneous cluster environments, including cloud, customer self-hosted infrastructure, and local NVIDIA GPU deployments.

You will design an efficient, scalable, and reliable automation factory infrastructure that transforms AI models into NIMs validated for best-in-class performance and accuracy. The work includes building factory infrastructure, pipelines, backends, Docker builds, test harnesses, metrics, performance engineering, log ingestion, and related capabilities.

Responsibilities

Develop a factory pipeline that takes an AI model and produces a deployable service validated across cloud, on-premises, and Kubernetes environments.
Define and deliver rapid iterations on technical strategies and roadmaps for the NIM factory.
Design interfaces, data models, schemas, and observability capabilities for the factory pipeline and its compute infrastructure.
Design and develop scalable and reliable factory components with technical leaders.
Collaborate with multiple AI model teams to understand requirements and build infrastructure that improves team productivity.
Define metrics and drive improvements based on user feedback.
Mentor colleagues and collaborate across teams.
Lead and contribute to multifunctional technical efforts.
Debug and analyze the performance of distributed microservices and cloud systems.

Requirements

Advanced programming experience building distributed and compute systems, backend services, microservices, and cloud technologies.
Experience working with multifunctional teams, principals, and architects across organizational boundaries.
Experience mentoring team members and adapting direction and expectations to customer needs.
Deep technical expertise in distributed containerized applications using technologies such as Docker, Kubernetes, Cloud Endpoints, Helm, and Prometheus.
Passion for building rich microservice applications and build and test automation pipelines.
Excellent interpersonal skills and the ability to lead multifunctional efforts.
A bachelor's or master's degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
At least 8 years of experience developing performant microservices, cloud software, and/or tooling.

Preferred Qualifications

Experience delivering event-driven applications using services such as Temporal, Kafka, Redis, or similar technologies, with the ability to discuss the advantages and disadvantages of these choices.
Experience building and deploying containers for microservice, cloud, and on-premises deployments and their associated CI/CD pipelines.
Prior experience with large-scale full-stack development.

Benefits

Eligibility for equity and benefits.
NVIDIA is an equal opportunity employer committed to an inclusive work environment.
Applications will be accepted at least until November 6, 2026.
NVIDIA uses AI tools in its recruiting processes.