Senior Perception Engineer, Obstacle Foundation Models - Autonomous Vehicles

Details

NVIDIA is seeking an exceptional Senior Perception Engineer to help design and productize its next-generation autonomous driving perception stack. The role focuses on the core 3D obstacle perception pipeline, including architecture and algorithm design, hands-on implementation, and modern transformer-based, multi-modal, and vision-language techniques.

Responsibilities

• Develop and improve the technical design, architecture, and roadmap for 3D obstacle perception supporting end-to-end autonomous driving functionalities, leveraging CNN and transformer-based architectures where appropriate.
• Design and implement advanced 3D perception models using multi-camera inputs and/or multi-sensor fusion, including camera, radar, and lidar, for obstacle detection and tracking.
• Explore BEV and transformer-based 3D perception approaches.
• Build efficient, production-grade deep learning models by defining objectives, selecting and prototyping architectures, running experiments, and applying best practices for training and evaluation.
• Apply techniques such as large-scale pretraining, distillation, and parameter-efficient fine-tuning, including LoRA.
• Define and maintain KPI frameworks to quantify perception performance.
• Analyze large-scale real and synthetic datasets to identify failure modes and systematically improve accuracy, robustness, and efficiency.
• Incorporate self-supervised learning and representation learning approaches when beneficial.
• Contribute to the perception data strategy by specifying data and labeling requirements, prioritizing data collection and annotation, and collaborating with data and ground-truth teams.
• Develop model-assisted workflows such as active learning, auto-labeling, vision-language models, and model-in-the-loop tooling.
• Collaborate with safety, systems, and software teams to ensure perception solutions meet requirements for safety, latency, resource usage, software robustness, and large-scale deployment.

Requirements

• PhD with 4+ years, MS with 6+ years, or BS or equivalent experience with 8+ years of relevant experience in Computer Science, Computer Engineering, or a related technical field.
• Hands-on experience developing deep learning-based perception or closely related systems for complex real-world problems.
• Strong proficiency with frameworks such as PyTorch and a track record of taking models from prototype to production.
• Experience with data-driven development, including collaboration with data, labeling, and ground-truth teams on data strategy, labeling quality, and iterative model improvement.
• Strong programming skills in Python and/or C++, with experience building reliable, high-performance, production-quality software.
• Excellent communication and collaboration skills, with the ability to work effectively across multidisciplinary teams.

Preferred Qualifications

• Experience designing and deploying perception solutions for autonomous driving or robotics using camera-based deep learning at scale.
• Experience architecting and deploying DNN-based perception pipelines on embedded or real-time platforms, including optimization for latency, memory, and compute constraints.
• Familiarity with CNNs, transformers, large-scale pretraining, parameter-efficient fine-tuning such as LoRA, and vision-language models.
• Strong publication record or recognized contributions in deep learning, computer vision, or autonomous systems at conferences or journals such as CVPR, ICCV, NeurIPS, or IROS.
• Deep understanding of 3D computer vision fundamentals, including camera modeling and calibration, intrinsic and extrinsic parameters, multi-view geometry, and 3D representations.
• Experience applying 3D computer vision concepts in transformer-based 3D or BEV perception pipelines.
• Experience with CUDA development and optimizing training or inference pipelines through custom CUDA kernels or other GPU-accelerated components.

Compensation and Additional Information

The base salary is determined by location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD–287,500 USD for Level 4 and 224,000 USD–356,500 USD for Level 5. The role is also eligible for equity and benefits.

Applications will be accepted at least until May 25, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes and is an equal opportunity employer.