Performance Engineer, Inference Engine

Details

Anthropic is seeking a Performance Engineer to build and optimize its inference engine, the software layer between accelerator kernels and the routing layer. The system manages request batching, model placement across chips, memory for weights and activations, forward-pass coordination, and model state across requests. It runs across Anthropic's accelerator platforms, serving Claude and supporting research workloads.

The role focuses on improving throughput, cost, reliability, and latency across accelerator and cloud platforms. The engineer will work across accelerator programming, high-performance host-device systems, and large-scale distributed systems, with an emphasis on understanding hardware and bandwidth constraints including FLOPs, HBM, PCIe, RDMA, and network links. Familiarity with transformer architecture is beneficial.

Responsibilities

• Keep accelerator utilization high by minimizing overhead and waiting.
• Reuse cached model state where it is more efficient than recomputation.
• Build observability, profile systems, model performance impacts, deploy improvements, and measure results iteratively.
• Maintain model quality and reliability across platforms and over time.
• Support production safety systems through efficient and robust inference infrastructure.
• Collaborate with safeguards and safety teams.

Requirements

• Working mental model of LLM inference, including how prefill and decode use accelerator compute, memory, and interconnect resources, and what the host performs concurrently.
• Ability to quickly learn unfamiliar systems and ship consequential changes.
• Strong systems programming skills in Rust, C++, or a similar language, with attention to code quality and testing.
• Analytical approach to performance optimization: observe and profile, form hypotheses, test, modify code, and measure again.
• Collaborative, low-ego working style and willingness to support work outside the formal job description.
• Interest in pair programming and the societal impacts of the work.

Preferred Qualifications

• Experience working inside an LLM serving engine and understanding where its abstractions become limiting.
• GPU or accelerator programming experience.
• Knowledge of operating system internals.
• Experience with transformer-based language modeling.
• Experience building an allocator, cache, scheduler, or high-bandwidth transport.
• Fluency in Rust.
• Experience making systems reproducible through determinism, replay, and property-based testing.

Education and Logistics

• Bachelor's degree or equivalent combination of education, training, and experience.
• Relevant field of study demonstrated through coursework, training, or professional experience.
• Years of experience will correlate with the internal job level requirements.
• Hybrid policy: staff are expected to work from an Anthropic office at least 25% of the time, though some roles may require more office time.
• Anthropic sponsors visas where possible and makes reasonable efforts to support visa applications, with assistance from an immigration lawyer.

Compensation

Annual salary: $350,000–$850,000 USD.

Anthropic offers competitive compensation and benefits, optional equity donation matching, generous vacation and parental leave, flexible working hours, and office collaboration spaces.