Staff+ Software Engineer, ML Sampling Path

Details

The Safeguards ML Sampling Path team builds and operates the production services that power Claude's safety systems. These services sit on the token generation path across every platform Claude runs on, so every request must pass through them and each millisecond of added latency affects users.

Responsibilities

• Design, build, and operate backend systems that process every token on the generation path for Claude requests, including the streaming contract with the API and inference engines.
• Own latency and reliability end to end by defining and maintaining SLOs and error budgets for added latency, time-to-first-token, and availability.
• Lead incident response and postmortem follow-through.
• Ship changes to the hot path rapidly but safely through canaried and gradual rollouts, error-budget and latency gating, and fast rollbacks.
• Drive per-token performance, reduce tail latency, and keep costs flat as traffic, models, and checks per request grow.
• Set technical direction for the sampling path, lead design reviews, make latency, reliability, and cost trade-off decisions with inference and research teams, mentor engineers, and raise the operational bar across the Safeguards organization.

Requirements

• Experience designing, building, and operating high-QPS systems at global scale with production accountability, including incident response, outages, and postmortem-driven remediation.
• Strong foundation in distributed systems, including replication, consistency trade-offs, failure modes, and SLO management under load.
• Experience designing systems for graceful degradation in response to slow dependencies, dropped streams, and partially rolled-out deployments.
• Experience shipping broad or all-encompassing changes to mission-critical systems, such as database migrations, interface changes, or rewrites.
• Strong candidates may have 8+ years of industry software engineering experience.
• Familiarity with LLM inference systems and transformer-based models is a plus, but not required.
• Minimum education: bachelor's degree or an equivalent combination of education, training, and/or experience.
• Required field of study: a field relevant to the role as demonstrated through coursework, training, or professional experience.

Benefits

Anthropic offers competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office space for collaboration. Anthropic sponsors visas, although sponsorship may not be available for every role or candidate.

The role follows a location-based hybrid policy. Staff are currently expected to work from one of Anthropic's offices at least 25% of the time, though some roles may require more office time.