Research Engineer, Tokens (Pre-Training)

Details

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. The team consists of researchers, engineers, policy experts, and business leaders working to build beneficial AI systems.

Responsibilities

• Conduct pretraining data research.
• Understand pretraining data trends and scaling laws.
• Optimize pretraining data mixes.
• Investigate potential new sources of data.
• Build research tools to better understand experimental results.
• Process and utilize pretraining data effectively.

Requirements

• Significant software engineering experience.
• Results oriented with flexibility and impact focus.
• Comfortable working in an empirical research environment.
• Care about societal impacts of their work.

Strong candidates may also have experience with:

• High performance, large-scale ML systems.
• Language modeling with transformers.
• Large-scale ETL.
• Designing ML experiments and researching ML fundamentals.
• Inspecting and iterating on data (e.g., ML competitions, Quantitative Finance).

Representative projects

• Comparing compute efficiency of different datasets.
• Creating multimodal datasets in consumable formats for models.
• Scaling data processing jobs to thousands of machines.
• Designing research tools for data ablation experiments.
• Creating interactive visualizations of semantic clusters in training data.

Logistics

• Requires at least a Bachelor's degree in a related field or equivalent experience.
• Hybrid location policy: staff expected to be onsite at least 25% of the time; some roles may require more.
• Visa sponsorship available with immigration lawyer support.

Benefits

• Competitive compensation and benefits.
• Optional equity donation matching.
• Generous vacation and parental leave.
• Flexible working hours.
• Collaborative office space in San Francisco.

Company culture

• Focus on large-scale AI research with high impact.
• Team values communication and empirical research.
• Emphasis on ethical and societal impacts of AI work.
• Encourages diverse perspectives and inclusive hiring.