Staff + Senior Software Engineer, Cloud Inference Launch Engineering

Details

Anthropic's Cloud Inference team scales and optimizes Claude for developers and enterprise companies across AWS, GCP, Azure, and future cloud service providers. The team owns Claude's end-to-end product experience on each cloud platform, including API integration, intelligent request routing, inference execution, capacity management, and operations.

The model and inference launch team owns the validation pipeline for the inference server and load balancer across these platforms. This includes ensuring that model launches, performance improvements, and safeguard integrations reach cloud platforms with correctness, performance, and reliability intact.

Responsibilities

• Be on the critical path for frontier model launches by bringing up inference for new model architectures and shipping them to cloud platforms in lockstep with Anthropic's first-party platform.
• Work with the core inference team to bring new inference features, such as structured sampling and prompt caching, to cloud platforms, owning the platform-specific integration required for production.
• Identify and resolve differences between first-party and cloud service provider inference, including configuration drift, observability, deployment patterns, and difficult cross-platform bugs.
• Design, build, and own CI/CD infrastructure for the inference server and load balancer across cloud platforms.
• Implement shadow traffic, performance baselines for throughput and latency, and correctness checks to catch regressions before production.
• Reduce merge-to-production cycle time by making validation faster, more parallel, and cost-effective enough to run on constrained accelerator pools without compromising reliability.
• Analyze observability data across providers to identify performance bottlenecks, cost anomalies, and regressions, and drive remediation using real-world production workloads.

Requirements

• Strong interest in LLM serving; prior inference or machine learning experience is not required.
• Significant software engineering experience, with a strong background in high-performance, large-scale distributed systems serving millions of users.
• Track record of building automation or test infrastructure that measurably improved release velocity or reliability.
• Experience building or operating services on at least one major cloud platform: AWS, GCP, or Azure.
• Exposure to Kubernetes, infrastructure as code, or container orchestration.
• Ability to collaborate cross-functionally with internal teams and external partners.
• Ability to quickly learn new technologies, hardware platforms, and provider ecosystems.
• Highly autonomous approach and ownership of problems end-to-end, including work outside the formal job description.

Preferred Qualifications

• LLM inference optimization, batching, and caching strategies.
• Capacity-constrained scheduling or shared-resource test infrastructure.
• Solid understanding of multi-region deployments, request routing, load balancing, and global traffic management.
• Experience working with cloud service provider partner teams to scale infrastructure across multiple platforms, including differences in networking, security, privacy, and managed services.
• Proficiency in Python or Rust.

Education and Experience

• Bachelor's degree or an equivalent combination of education, training, and experience.
• A field relevant to the role, as demonstrated through coursework, training, or professional experience.
• Required years of experience correlate with the internal job level requirements for the position.

Benefits

Anthropic offers competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and an office space for collaboration.

Additional Information

Anthropic currently expects staff to work from one of its offices at least 25% of the time, although some roles may require more office time. Anthropic sponsors visas and states that it makes every reasonable effort to obtain a visa for candidates when an offer is made, with assistance from an immigration lawyer. Anthropic is a public benefit corporation headquartered in San Francisco.