Senior Machine Learning Engineer, AI Safety Evaluations

Details

NVIDIA is seeking a Machine Learning Engineer, AI Researcher, or Data Scientist to support AI Safety and Responsibility efforts within Enterprise Risk Management. The role focuses on applying machine learning, data science, and statistical expertise to develop rigorous evaluations, benchmarks, experiments, and tools for understanding how multimodal large language models behave under pressure. The work will support improved training methods and safeguards, with reproducible results informing technical reviews and executive decisions.

Responsibilities

• Design and maintain safety benchmarks for large language and multimodal models by translating risk scenarios into test cases, datasets, and scoring rubrics.
• Define metrics and protocols that assess safety alongside model usefulness, using baselines, uncertainty estimates, and failure analysis to distinguish meaningful improvements from noise.
• Build reusable Python pipelines with versioned datasets, model configurations, and experiment-tracking workflows to reproduce results and compare performance across model releases.
• Develop automated red-teaming tools to discover failure modes and expand evaluation coverage, incorporating human review to validate automated scoring.
• Investigate model failures with researchers, document actionable findings, and develop regression tests to determine whether model changes resolve identified issues.
• Partner with platform engineers to store and aggregate evaluation results.
• Clearly communicate findings, limitations, trade-offs, and emerging risks to researchers, engineering partners, and decision-makers.

Requirements

• Bachelor’s degree or equivalent experience.
• At least 6 years of experience in machine learning engineering or evaluation.
• Experience developing benchmarks, metrics, and statistically sound comparisons for machine learning systems, including selecting baselines and interpreting uncertainty.
• Ability to translate ambiguous evaluation questions into testable experiments.
• Strong Python skills and experience building maintainable shared tools, data pipelines, and experiment-tracking workflows.
• Ability to assess dataset quality, identify limitations in metrics and scoring methods, and support conclusions with reproducible evidence.
• Strong communication skills, including the ability to explain findings and trade-offs to researchers, engineering partners, and executives.

Preferred Qualifications

• Experience evaluating LLMs, multimodal models, or agentic systems through adversarial testing or automated red-teaming.
• Contributions to public benchmarks or evaluation frameworks.
• A publication track record and interest in continuing to publish at top-tier venues.
• Experience validating model-based evaluators against human judgments or designing human-in-the-loop evaluation workflows.
• Strong model evaluation skills and a commitment to sound statistical practices, including the use of confidence intervals.

Compensation and Benefits

The expected base salary ranges are USD 184,000–287,500 for Level 4 and USD 224,000–356,500 for Level 5. The position also includes eligibility for equity and benefits. Applications will be accepted at least until November 9, 2026. NVIDIA is an equal opportunity employer.