Research Engineer, Model Performance & Quality

Details

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. The Model Performance team focuses on systematically understanding and monitoring model quality in real-time. This Research Engineer role blends research and engineering: training production models, developing monitoring systems, and creating novel evaluation methodologies to measure real-world model utility and safety.

Responsibilities

• Design and implement monitoring infrastructure to observe how model behaviors evolve throughout training and in production.
• Develop next-generation evaluation frameworks that move beyond traditional benchmarks to capture real-world utility.
• Build automated quality assessment pipelines, including custom classifiers to continuously monitor RL transcripts for complex issues.
• Train production models and collaborate with research teams to translate cutting-edge evaluation techniques into production-ready systems.
• Work with engineering teams to ensure monitoring infrastructure scales with increasingly complex training workflows.
• Debug complex, distributed systems and reason about failure modes in training and deployment.

Requirements

• Proficiency in Python and experience building production ML systems.
• Experience with training, evaluating, or monitoring large language models (LLMs).
• Strong analytical skills for interpreting training metrics and model behavior.
• Ability to balance research exploration with engineering rigor and collaborate across teams.
• Bachelor's degree in a related field or equivalent experience (required).

Strong candidates may have

• Experience with reinforcement learning and language model training pipelines.
• Experience designing and implementing evaluation frameworks or benchmarks.
• Background in production monitoring, observability, and incident response.
• Experience with statistical analysis and experimental design.
• Knowledge of AI safety and alignment research.

Strong candidates need not have

• Formal certifications or education credentials beyond the required degree/equivalent experience.
• Academic research experience or publication history.
• Prior experience specifically in AI safety or evaluation.

Logistics

• Location-based hybrid policy: staff are expected to be in one of Anthropic's offices at least 25% of the time (some roles may require more time in office).
• Visa sponsorship: Anthropic does sponsor visas and will make reasonable efforts and provide immigration support for offers where sponsorship is required, though sponsorship may not be possible for every role or candidate.

Compensation

• Annual salary range: $315,000 - $340,000 USD.

Why work here / Benefits

• Opportunity to shape model quality assessment approaches used across the field and directly impact the safety and reliability of deployed AI systems.
• Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and collaborative office environments.

Anthropic encourages applicants even if they do not meet every listed qualification and values diverse perspectives in AI research and engineering.