Manager, Software Engineering - Production AI Inference

Details

NVIDIA is the platform upon which every new AI-powered application is built. This role is a hands-on engineering management position to lead production AI inference for NVIDIA Inference Microservices (NIM) — the production runtime through which customers deploy optimized, enterprise-supported AI inference across cloud, data center, and edge environments. NIM combines optimized inference engines, model profiles/recipes, validated runtime configurations, and security hardening. The role leads the team accountable for turning fast-moving model and inference engine work into reliable NIM releases that customers can operate with confidence.

Responsibilities

• Lead the team responsible for shipping production-ready LLM NIMs, including planning, new model onboarding, validated serving recipes, release readiness, and post-release follow-through.
• Build a predictable operating model for the team through roadmap planning, a weekly execution rhythm, launch checklists, clear ownership boundaries, collaborator communication, and issue management.
• Own project execution by anticipating schedule, staffing, and dependency risks; adapt plans under pressure and collaborate with peer managers to dynamically prioritize engineering timelines.
• Drive continuous improvement in production workflows through RCCA and partner feedback, removing unnecessary and redundant work while keeping the team focused on production outcomes.
• Build and maintain a world-class AI inference engineering team by fostering an innovative culture, setting clear expectations, maintaining active feedback loops, and mentoring engineers and emerging leaders.

Requirements

• 10+ overall years building production software, including 3+ years of managing software engineering teams.
• Experience delivering production software with strong quality, reliability, and release expectations.
• Experience driving process improvements and improving operational efficiency.
• Excellent communication and collaborator management; ability to influence executive leadership across product, research, security, and operations.
• Deep understanding of AI/ML fundamentals, innovative model architectures, inference engine/kernel, performance optimization strategies, accelerated computing, large-scale distributed systems, and security hardening.
• A degree in Computer Science, Computer Engineering, or a related field (BS or MS) or equivalent experience.

Ways to stand out from the crowd

• Built and managed globally distributed organizations; established durable engineering processes that significantly improved quality and velocity across multiple teams.
• Recognized industry leader with contributions to open-source ecosystems (e.g., vLLM, SGLang, TensorRTLLM, Dynamo, Triton, PyTorch), technical publications, or talks in containers, Kubernetes, GPU, or inference communities.
• Drove measurable performance improvements for large-scale LLM inference systems, including latency, throughput, GPU utilization, cost efficiency, and performance regression prevention across production releases.
• Hands-on experience with core GPU technologies such as CUDA, cuDNN, CUTLASS, cuBLAS, NCCL, NIXL, NVLink, and GPUDirect RDMA.
• Hands-on experience delivering enterprise or government-ready AI software, including FedRAMP, air-gapped deployments, regulated environments, security hardening, compliance evidence, and production support expectations.

Compensation & Benefits

• Base salary range is 224,000 USD - 356,500 USD for Level 3, and 272,000 USD - 431,250 USD for Level 4.
• Eligible for equity and company benefits.

Additional notes

• Applications for this job will be accepted at least until July 6, 2026.
• NVIDIA uses AI tools in its recruiting processes. NVIDIA is an equal opportunity employer and committed to an inclusive work environment.