Senior Deep Learning Engineer, Accuracy Evaluation

Details

We are seeking senior engineers to pioneer new methodologies for accurately assessing the performance and capabilities of groundbreaking deep learning models, including LLMs, RAG, agents, and vision models. You will collaborate across the organization to bring the latest flagship models from NVIDIA's community and partners to life. This role offers an opportunity to help shape the future of AI at a fast-growing company at the forefront of the AI revolution.

You will join a team of software engineers and partners delivering advanced models with high-speed inference. The role involves working on enterprise-grade GPU clusters capable of hundreds of PetaFLOPS and gaining early access to unreleased hardware, directly impacting NVIDIA's roadmap and the broader AI landscape.

Responsibilities

• Design and build decision-grade evaluation environments for NVIDIA's frontier models spanning reasoning, multimodal, long-context, and agentic systems, producing auditable accuracy signals that gate major model releases.
• Research and develop novel evaluation methodologies for emerging model families and capability domains, including low-precision numerics, multi-turn agentic tasks, and code generation, where established benchmarks do not yet exist or do not generalize.
• Build and operate evaluation infrastructure and pipelines, including benchmark environments, regression CI systems, and statistical analysis tooling used by model, product, and applied research teams across NVIDIA.
• Partner with model research, training, and customer teams to translate evaluation signals into concrete decisions, including release go/no-go decisions, training iteration direction, and competitive positioning against external frontier models.

Requirements

• BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field.
• At least 6 years of hands-on experience with LLMs, including designing and running evaluations for large language models or multimodal AI systems.
• Experience with agentic, multi-turn, or reasoning-heavy settings.
• Strong statistical foundations, including experimental design, significance testing, regression analysis, and distinguishing signal from noise in benchmark results at scale.
• Proven experience building evaluation infrastructure, including pipelines, benchmark harnesses, and reproducible CI systems, rather than only consuming existing benchmarks.
• Clear and precise communication skills, with the ability to translate quantitative evaluation results into decisions for researchers, product teams, and senior leadership.

Preferred Qualifications

• Deep familiarity with open-source evaluation frameworks.
• Experience designing evaluations for agentic systems, including tool use, multi-turn reasoning, environment-based benchmarks such as SWE-bench, GAIA, and WebArena-style evaluations, or interactive evaluation settings.
• A track record of publishing or contributing to evaluation research, new benchmark design, methodology papers, or reproducibility analyses that shaped how the field measures model capability.
• Experience measuring model accuracy under low-precision inference, including FP8, INT4, and quantization-aware settings, with an understanding of how calibration and sparsity interact with benchmark results.
• Experience running large-scale workloads on HPC/Slurm clusters, including reproducible experiment management with MLflow or Weights & Biases and compute cost optimization across hundreds of benchmark runs.

Compensation

For Poland, the base salary range is 375,000 PLN–650,000 PLN per year.

NVIDIA is committed to fostering a diverse work environment and is an equal opportunity employer. The company does not discriminate, including in hiring and promotion practices, on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status, or any other characteristic protected by law.