Staff Software Engineer, Inference

Details

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. The Inference team builds and maintains the critical systems that serve Claude to millions of users worldwide. This team owns the full stack for inference: intelligent request routing, fleet-wide orchestration across diverse AI accelerators, and compute-agnostic large-scale inference deployments. The team’s dual mandate is maximizing compute efficiency for customer growth while enabling research by providing high-performance inference infrastructure to scientists.

As a Staff Software Engineer on the Inference team, you will work end-to-end to identify and address infrastructure blockers to serve Claude at scale while enabling research. The role involves working on distributed systems challenges across multiple accelerator families and cloud platforms, optimizing performance, and building orchestration and routing systems.

Responsibilities

• Design and implement intelligent routing algorithms that optimize request distribution across thousands of accelerators.
• Build and operate fleet-wide orchestration and autoscaling systems to dynamically match supply with demand for production, research, and experimental workloads.
• Implement production-grade deployment pipelines for releasing new models to millions of users.
• Integrate new AI accelerator platforms and maintain hardware-agnostic inference infrastructure.
• Contribute to inference features such as structured sampling and prompt caching.
• Support inference for new model architectures and multi-accelerator deployments.
• Analyze observability data and tune performance based on real-world production workloads.
• Manage multi-region deployments and geographic routing for global customers.

Requirements

• Significant software engineering experience, particularly with distributed systems and large-scale services.
• Familiarity with performance optimization, large-scale distributed systems, service orchestration, and intelligent request routing.
• Experience (or strong familiarity) with implementing and deploying machine learning systems at scale.
• Experience with load balancing, request routing, or traffic management systems.
• Familiarity with LLM inference optimization, batching strategies, and caching strategies is highly encouraged but not strictly required.
• Experience with Kubernetes and cloud infrastructure (AWS, GCP) is preferred.
• Proficiency in Python or Rust is desirable.
• Bachelor’s degree in a related field or equivalent experience is required.
• Results-oriented, flexible, and able to work across responsibilities that may extend beyond a narrow job description.
• Interest in machine learning systems and the societal impacts of AI.

Logistics

• Location: Dublin, Ireland. Location-based hybrid policy: staff are expected to be in one of Anthropic’s offices at least 25% of the time (some roles may require more time in office).
• Visa sponsorship: Anthropic may sponsor visas and retains immigration counsel to assist, though sponsorship is not guaranteed for every role/candidate.
• Applications reviewed on a rolling basis; no stated deadline.

Compensation

• Annual base salary range: €295,000 - €355,000 EUR. Total compensation may include equity, benefits, and incentive compensation.

Why Anthropic / How we work

• Anthropic values impact-driven, collaborative research and software development. The company emphasizes communication, large-scale "big science" research directions, and ethical considerations in AI development.

Benefits (high level)

• Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office workspace. Candidate AI usage guidance is provided for the application process.