Staff Software Engineer, Inference

Details

Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. The Inference team builds and maintains the systems that serve Claude to millions of users worldwide, managing the stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators. The team focuses on maximizing compute efficiency while enabling research by providing high-performance inference infrastructure across multiple accelerator families and cloud platforms.

Responsibilities

• Build and maintain large-scale, compute-agnostic inference deployments that serve production and research workloads.
• Design and implement intelligent request routing, load balancing, and traffic management to optimize request distribution across thousands of accelerators.
• Work on fleet orchestration, autoscaling compute fleets, and multi-region deployments.
• Integrate new AI accelerator platforms and maintain hardware-agnostic inference capabilities.
• Improve inference performance through batching strategies, caching, and other optimization techniques.
• Collaborate with researchers to enable new model architectures and inference features (e.g., structured sampling, prompt caching).
• Analyze observability data to tune performance and reliability for real-world production workloads.

Requirements

• Significant software engineering experience, particularly with distributed systems and performance optimization.
• Familiarity with large-scale service orchestration, load balancing, request routing, and traffic management systems.
• Experience implementing and deploying machine learning systems at scale is strongly desired.
• Familiarity with LLM inference optimization, batching, and caching strategies is highly encouraged.
• Experience with Kubernetes and cloud infrastructure (AWS, GCP) is valuable.
• Proficiency in Python or Rust is desirable.
• Bachelor’s degree in a related field or equivalent experience is required.

Representative projects

• Designing intelligent routing algorithms to optimize request distribution across thousands of accelerators.
• Autoscaling the compute fleet to match supply with demand across production, research, and experimental workloads.
• Building production-grade deployment pipelines for releasing new models to millions of users.
• Integrating new AI accelerator platforms to retain a hardware-agnostic advantage.
• Contributing inference features such as structured sampling and prompt caching.
• Managing multi-region deployments and geographic routing for global customers.

Compensation

• Annual salary range: €295,000 - €355,000 EUR

Logistics

• Location: Dublin, Ireland.
• Location-based hybrid policy: staff are expected to be in one of Anthropic's offices at least 25% of the time (role-level expectations may vary).
• Education requirement: at least a Bachelor’s degree in a related field or equivalent experience.
• Visa sponsorship: Anthropic states they sponsor visas and retain an immigration lawyer; they will make reasonable efforts to secure a visa if an offer is made.

Benefits

• Competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office space for collaboration.

How we work

• Anthropic emphasizes collaborative, high-impact research efforts, values communication skills, and encourages applicants from diverse backgrounds to apply.