Research Infrastructure Engineer, Training Systems

Details

About The Team

The team works on research and systems that advance frontier models. Work often goes beyond standard training recipes, building the infrastructure needed to make new training approaches practical at scale. Systems work is directly tied to research progress: better tools, abstractions, and runtimes can unlock experiments that would otherwise be too slow, brittle, or difficult to express.

About The Role

This is a systems engineering role focused on ML training infrastructure. You will work on the systems layer that turns novel research ideas into runnable, measurable training workloads for large models. The work can sit on the critical path for model releases, bringing both the excitement of direct impact and the responsibility of building systems that remain reliable under real pressure.

Responsibilities

• Build and maintain infrastructure for large-scale model training and experimentation.
• Design APIs and interfaces that make complex training workflows easier to express and harder to misuse.
• Improve reliability, debuggability, and performance across training and data pipelines.
• Debug issues spanning Python, PyTorch, distributed systems, GPUs, networking, and storage.
• Write tests, benchmarks, and diagnostics that catch meaningful regressions.

Requirements / Qualifications

• Experience working on ML training infrastructure, large-scale model training, or related systems.
• Strong systems instincts with a focus on performance, reliability, and clean abstractions.
• Good taste in API and interface design and empathy for researchers and engineers using your tools.
• Comfortable working across ML research code and production-quality infrastructure.
• Skilled at debugging from evidence: profiles, traces, logs, tests, and minimal reproductions.
• Familiarity with Python and PyTorch as they relate to training workloads.

Benefits

• Base pay range listed separately; total compensation includes equity and possible performance-related bonuses.
• Medical, dental, and vision insurance with employer HSA contributions.
• Pre-tax accounts (Health FSA, Dependent Care FSA, commuter benefits).
• 401(k) retirement plan with employer match.
• Paid parental leave (up to 24 weeks for birth parents, 20 weeks for non-birthing parents), paid medical and caregiver leave (up to 8 weeks).
• Flexible PTO for exempt employees; up to 15 days annually for non-exempt employees.
• 13+ paid company holidays and multiple coordinated office closures; paid sick or safe time as required by law.
• Mental health and wellness support; employer-paid basic life and disability coverage.
• Annual learning and development stipend.
• Daily meals in offices and meal delivery credits as eligible.
• Relocation support for eligible employees.
• Additional taxable fringe benefits (charitable donation matching, wellness stipends) as applicable.