GenAI Algorithms Intern – Model Optimization for Inference

Details

NVIDIA's Algorithmic Model Optimization Team focuses on optimizing generative AI models, including large language models (LLMs), vision-language models (VLMs), and diffusion models, for maximal inference efficiency. Techniques include quantization, speculative decoding, sparsity, distillation, pruning, neural architecture search, and automated deployment strategies.

The team conducts applied research to improve model efficiency and develops NVIDIA Model Optimizer, an open-source model optimization platform used internally at NVIDIA and externally by research and engineering teams. The intern will help optimize state-of-the-art generative AI models for inference and deployment on NVIDIA's latest hardware platforms, focusing on ease of use, compute and memory efficiency, and accuracy-performance tradeoffs through software-hardware co-design.

The work spans multiple layers of the AI software stack, including NVIDIA Model Optimizer, NeMo/Megatron, TensorRT-LLM, PyTorch, Hugging Face, vLLM, and SGLang. The role may also involve GPU-level optimization, including custom kernel development with CUDA and Triton.

Responsibilities

• Help design and build modular, scalable model optimization software platforms that support diverse AI models and optimization techniques.
• Explore, develop, and integrate deep learning optimization algorithms, including quantization, speculative decoding, sparsity, distillation, pruning, and neural architecture search, into NVIDIA's AI software stack.
• Deploy optimized models into leading open-source inference frameworks and contribute specialized APIs, model-level optimizations, and features tailored to the latest NVIDIA hardware capabilities.
• Collaborate with NVIDIA teams on model optimization solutions for customer use cases, aiming for optimal end-to-end workflows and balanced accuracy-performance tradeoffs.
• Conduct GPU kernel-level profiling to identify hardware and software optimization opportunities, including efficient attention kernels, KV cache optimization, and parallelism strategies.

Requirements

• Pursuing a Master's or PhD in Computer Science, AI, Applied Mathematics, or a related field.
• Strong deep learning background and hands-on experience with language models.
• Solid foundation in algorithms and programming.
• Proficiency in Python, PyTorch, and modern machine learning frameworks and tools.
• Good written and verbal communication skills, with the ability to work independently and collaboratively in a fast-paced environment.

Preferred Qualifications

• Experience with model optimization and efficiency, such as quantization, distillation, or pruning.
• Contributions to PyTorch, JAX, vLLM, SGLang, or other machine learning training and inference frameworks.
• Hands-on experience training or fine-tuning generative AI models on large-scale GPU clusters.
• Knowledge of GPU architectures and compilation stacks, with the ability to analyze and debug end-to-end performance.
• Familiarity with NVIDIA deep learning SDKs, including Model Optimizer and NeMo.

Benefits

NVIDIA offers a comprehensive benefits package and is committed to fostering a diverse work environment and providing equal employment opportunities.

Internship pay is based on the position, location, year in school, degree, and experience.