Senior AI Software Engineer, GenAI Framework

Details

NVIDIA is hiring for the GenAI Frameworks team to develop and expand Megatron Core and the NeMo Framework — open-source, scalable, cloud-native frameworks for Large Language Models (LLM) and multimodal foundation model pretraining and post-training. The role focuses on enabling users to develop, train, and optimize models by designing and implementing distributed training algorithms, model parallel paradigms, model optimizations, robust APIs, and performant tooling and libraries. You will collaborate with internal partners, users, and the open-source community to design and implement highly optimized solutions.

Responsibilities

• Design and develop the GenAI open-source Megatron Core and NeMo Framework.
• Solve large-scale, end-to-end AI training and inference challenges across the full model lifecycle: orchestration, data pre-processing, training, tuning, and deployment.
• Work at the intersection of AI applications, libraries, frameworks, and the full software stack.
• Innovate and improve model architectures, distributed training algorithms, and model parallel paradigms.
• Accelerate foundation model training and fine-tuning with mixed precision recipes and next-generation NVIDIA GPU architectures.
• Perform performance tuning and optimizations of deep learning frameworks and software components.
• Research, prototype, and develop robust and scalable AI tools and pipelines.

Requirements

• MS, PhD, or equivalent experience in Computer Science, AI, Applied Math, or related fields, plus 5+ years of industry experience.
• Experience with AI frameworks (for example, PyTorch, JAX) and/or inference and deployment environments (for example, TRTLLM, vLLM, SGLang).
• Proficient in Python programming, software design, debugging, performance analysis, test design, and documentation.
• Strong record of working across multiple engineering initiatives and improving AI libraries with new innovations.
• Strong understanding of AI / deep-learning fundamentals and practical applications.

Ways to stand out

• Hands-on experience in large-scale AI training and deep understanding of compute system concepts (latency/throughput bottlenecks, pipelining, multiprocessing) with demonstrated performance analysis and tuning.
• Expertise in distributed computing, model parallelism, and mixed precision training.
• Prior experience with generative AI techniques applied to LLMs and multi-modal learning (text, image, video).
• Knowledge of GPU/CPU architecture and numerical software.
• Previous contributions to open-source deep learning frameworks.

Compensation and benefits

• Base salary ranges by level: Level 3: 148,000 USD - 235,750 USD; Level 4: 184,000 USD - 287,500 USD. Exact base salary will be determined based on location, experience, and pay of employees in similar positions.
• Eligible for equity and benefits (see NVIDIA benefits pages).

Additional information

• Location: Santa Clara, CA, United States.
• Employment type: Full time.
• Applications accepted at least until October 3, 2025.
• NVIDIA is an equal opportunity employer committed to fostering a diverse work environment.