Senior Perception Engineer, Obstacle Foundation Models – Autonomous Vehicles

Details

NVIDIA is seeking an exceptional Senior Perception Engineer to help design and productize its next-generation autonomous driving perception stack. The role focuses on the core 3D obstacle perception pipeline, architecture and algorithm design, and hands-on implementation of modern transformer-based, multimodal, and vision-language techniques.

Responsibilities

• Develop and improve the technical design, architecture, and roadmap for 3D obstacle perception supporting end-to-end autonomous driving functionalities, leveraging CNN and transformer-based architectures where appropriate.
• Design and implement advanced 3D perception models using multi-camera inputs and/or multi-sensor fusion, including camera, radar, and lidar, for obstacle detection and tracking.
• Explore BEV and transformer-based 3D perception approaches.
• Build efficient, production-grade deep learning models by defining objectives, selecting and prototyping architectures, running experiments, and applying best practices for training and evaluation.
• Apply techniques such as large-scale pretraining, knowledge distillation, and parameter-efficient fine-tuning, including LoRA.
• Define and maintain KPI frameworks to quantify perception performance.
• Analyze large-scale real and synthetic datasets to identify failure modes and improve accuracy, robustness, and efficiency.
• Incorporate self-supervised learning and representation learning approaches where beneficial.
• Contribute to perception data strategy, including data and labeling requirements, data collection and annotation prioritization, and collaboration with data and ground-truth teams.
• Develop model-assisted workflows using active learning, auto-labeling, vision-language models, and model-in-the-loop tooling.
• Collaborate with safety, systems, and software teams to ensure perception solutions meet requirements for safety, latency, resource usage, software robustness, and large-scale deployment.

Requirements

• PhD with 4+ years, MS with 6+ years, or BS or equivalent experience with 8+ years of relevant experience in Computer Science, Computer Engineering, or a related technical field.
• Hands-on experience developing deep learning-based perception or closely related systems for complex real-world problems.
• Strong proficiency with frameworks such as PyTorch and a track record of taking models from prototype to production.
• Experience in data-driven development, including collaboration with data, labeling, and ground-truth teams on data strategy, labeling quality, and iterative model improvement.
• Strong programming skills in Python and/or C++, with experience building reliable, high-performance, production-quality software.
• Excellent communication and collaboration skills, with the ability to work effectively across multidisciplinary teams.

Preferred Qualifications

• Experience designing and deploying perception solutions for autonomous driving or robotics using camera-based deep learning at scale.
• Experience architecting and deploying DNN-based perception pipelines on embedded or real-time platforms, including optimization for latency, memory, and compute constraints.
• Experience with CNNs, transformers, large-scale pretraining, parameter-efficient fine-tuning such as LoRA, or vision-language models.
• Strong publication record or recognized contributions in deep learning, computer vision, or autonomous systems at leading conferences or journals such as CVPR, ICCV, NeurIPS, or IROS.
• Deep understanding of 3D computer vision fundamentals, including camera modeling and calibration, intrinsic and extrinsic parameters, multi-view geometry, and 3D representations.
• Experience applying 3D computer vision concepts in transformer-based 3D or BEV perception pipelines.
• Experience with CUDA development and optimizing training or inference pipelines through custom CUDA kernels or other GPU-accelerated components.

Compensation and Additional Information

The base salary is determined based on location, experience, and the pay of employees in similar positions. The base salary ranges are:

• Level 4: $184,000–$287,500 USD per year
• Level 5: $224,000–$356,500 USD per year

The role also includes eligibility for equity and benefits. Applications will be accepted at least until July 24, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes and is an equal opportunity employer.