Senior Software Engineer - GPU Local AI Platforms

Details

NVIDIA's Local AI team is building the software stack that enables large language models and generative AI applications to run efficiently on NVIDIA edge AI hardware. The team owns the platform, including performance, CI/CD pipelines, validated recipes, and model bring-up infrastructure, allowing developers and partners to run LLMs reliably at scale.

Responsibilities

• Track and evaluate innovations in leading open-source LLM inference frameworks, identifying performance-critical features and algorithmic improvements relevant to NVIDIA edge AI hardware.
• Analyze how new model architectures and inference algorithms—including attention variants, mixture-of-experts routing, speculative decoding, multi-token prediction, and quantized inference—map onto NVIDIA GPU architecture. Identify mismatches, fallback paths, and optimization opportunities.
• Characterize multi-node inference behavior, including collective communication primitives such as NCCL and RCCL, topology-aware all-reduce strategies, and parallelism efficiency on edge cluster configurations.
• Produce performance analysis reports that map theoretical hardware limits, including memory bandwidth, FLOP/s, and interconnect throughput, to observed inference throughput, latency, and utilization.
• Own the model validation workflow for new model releases, including architecture compatibility assessment, inference recipe development, performance characterization, and publication to developer recipe sites.
• Develop and maintain developer-facing inference recipes, automate staleness detection, and build feedback loops from CI results to recipe updates.
• Engage with the community and partners on model bring-up questions and serve as the technical point of contact for hardware-specific inference issues.

Requirements

• BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.
• 12+ years of software engineering experience with depth in GPU computing, machine learning systems, or high-performance inference.
• Strong Python or C++ programming, software design, and software engineering skills.
• Hands-on experience with GPU kernel development or optimization using CUDA/C++, Triton, or an equivalent technology. Understanding of how thread blocks, memory hierarchy, and warp execution affect real-world performance is required.
• Working knowledge of LLM inference internals, including attention mechanisms, KV-cache management, continuous batching, quantization formats, and tensor parallelism.
• Container engineering expertise, including multi-architecture Docker or OCI builds, layer optimization, runtime configuration, and NVIDIA Container Toolkit.
• Strong analytical skills, including the ability to form performance hypotheses, design experiments, interpret results, and communicate findings clearly.

Benefits

NVIDIA offers competitive salaries, a comprehensive benefits package, equity, and benefits for employees and their families. The base salary range is USD 224,000–356,500 for Level 5 and USD 272,000–431,250 for Level 6. The salary is determined based on location, experience, and compensation for similar positions.

NVIDIA is committed to fostering an inclusive work environment and is an equal opportunity employer. The company uses AI tools in its recruiting processes.