Senior Deep Learning Scientist, Conversational AI

Details

NVIDIA is an industry leader with groundbreaking developments in High-Performance Computing, Artificial Intelligence and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. GPU deep learning ignited modern AI — the next era of computing — with the GPU acting as the brain of computers, robots, autonomous cars and conversational AI that can perceive and understand the world. Today, we are increasingly known as “the AI computing company.” We’re looking to grow our company, and build our teams. Join us at the forefront of technological advancement!

NVIDIA is looking for a Senior Deep Learning Scientist, Conversational AI who is passionate in areas such as, embodied AI, conversational AI, robotics (navigation, manipulation), AR/VR/MR, egocentric computer vision, grounded 3D perception, simulation and sim2real transfer, pre-training for embodied agents, and human-AI interaction, bringing to bear foundational knowledge from areas such as deep learning, reinforcement learning, computational statistics, and applied mathematics. You will have an opportunity to make core algorithmic advances and apply your ideas at scale using our NeMo LLM MLOps platform. You will develop high-impact, high-visibility Large language model products and improve the experience of millions of customers. If you’re creative & passionate about solving real-world embodied conversational AI problems, come join our Digital Human LLM team.

Responsibilities

• Develop, Train, Fine-tune, and Deploy LLMs for driving embodied conversational AI systems including multimodal understanding, speech synthesis, image generation, UI and animation rendering and control, environment interaction, and dialog reasoning and tool systems.
• Apply innovative fundamental and applied research to develop products for embodied conversational artificial intelligence.
• Build novel data-driven paradigms for embodied intelligence including customization recipes for different domains and enterprise use cases.
• Develop systems and framework using various data modalities (images, video, text, audio, tactile, etc.) and the roles they play in different levels of embodied reasoning and decision making.
• Explore paradigms that can deliver a spectrum of embodied behaviors - from simulated characters to real robots, and from short horizon, low level to long horizon, high level.
• Enable long-horizon reasoning and facilitate low level skills for Embodied AI tasks.
• Apply alignment techniques such as instruction tuning, reinforcement learning from human feedback (RLHF), and parameter efficient fine-tuning such as p-tuning, adaptors, LoRA, and so on to improve use cases.
• Measure and benchmark model and application performance and Analyze model accuracy and bias and recommend the next course of action & Improvements.
• Drive the gathering, building, and annotation of domain specific datasets to train LLMs for different embodied tasks and applications and maintain model evaluation systems and characterize performance and quality metrics across platforms for various AI and system components.
• Collaborate and innovate with various teams on new product features, improvements of existing products and participate in developing and reviewing code, design documents, use case reviews, and test plan reviews.

Requirements

• Master’s degree (or equivalent experience) or PhD in Computer Science, Electrical Engineering, Artificial Intelligence, or Applied Math with 5+ years of experience.
• Excellent programming skills in Python with strong fundamentals optimizations and software design.
• Solid understanding of ML/DL techniques, algorithms and tools with exposure to CNN, RNN (LSTM), Transformers (ViT, BERT, BART, GPT/T5, Megatron, LLMs).
• Hands-on experience on conversational AI Technologies.
• Experience with Training ViT, BERT, GPT and Megatron Models for different computer vision, NLP and dialog system tasks using “PyTorch” Deep Learning Frameworks and performing data wrangling and tokenization.
• Solid understanding of MLOps life cycle and experience with MLOps workflows & traceability and versioning of datasets including knowhow of database management and queries (in SQL, MongoDB etc).
• Strong collaborative and interpersonal skills, and optimally guide and influence within a dynamic matrix environment.

Ways to stand out from the crowd:

• Fluency in a non-English language - Spanish / Mandarin / German / Japanese / Russian / French / UK English / Arabic/ Korean / Italian / Portuguese.
• Familiarity with GPU based technologies like CUDA, CuDNN and TensorRT.
• Background with Dockers and Kubernetes and deploying machine learning models on data center, cloud, and embedded systems and strong C++ programming skills.
• Experience developing all aspects of large language models.
• Integrating embodied AI systems with various sensor inputs.

With competitive salaries and a generous benefits package, we are widely considered to be one of the technology world’s most desirable employers. We have some of the most forward-thinking and hardworking people in the world working for us and, due to outstanding growth, our best-in-class engineering teams are rapidly growing. If you’re a creative and autonomous engineer with a real passion for technology, we want to hear from you!