Software Engineer, Agent Optimization & Evaluation

Details

Reddit is building a next-generation AI developer platform: a best-in-class agent stack that helps engineers build, test, review, and ship software efficiently while preserving control over quality, costs, and long-term AI strategy. The optimization harness measures and improves agents across the software development lifecycle by benchmarking models, detecting regressions, optimizing routing, incorporating production feedback, and improving developer productivity.

The role reports to Kyle Maxwell and partners with Platform Engineering, Infrastructure, and Developer Productivity teams to shape AI-assisted software development across Reddit.

Responsibilities

• Design and build Reddit's agent optimization harness for evaluating, benchmarking, and improving AI agents across real software engineering workflows.
• Develop Reddit-specific evaluation environments that accurately measure agent performance against meaningful engineering tasks.
• Build systems for offline replay, automated evaluation, synthetic task generation, regression detection, and human-in-the-loop feedback.
• Design model routing and orchestration strategies that optimize for quality, latency, and cost.
• Define metrics and experimentation frameworks that measure developer productivity and engineering velocity.
• Build tooling to analyze agent behavior, investigate failures, and accelerate iteration.
• Partner with Platform Engineering, Infrastructure, and Developer Productivity teams to integrate agent capabilities into engineering workflows.
• Help shape Reddit's long-term strategy for a flexible, model-agnostic agent platform that provides control over AI capabilities and economics.

Requirements

• 5–12+ years of engineering experience.
• Extensive experience building and shipping large-scale production software systems.
• Experience designing evaluation, experimentation, benchmarking, or measurement systems for AI, search, developer platforms, or other complex distributed systems.
• Strong understanding of modern large language models, AI agents, and the engineering systems required to deploy them effectively.
• Experience building systems that optimize across multiple dimensions, including quality, latency, reliability, and cost.
• Excellent software engineering fundamentals and a passion for building production-quality systems.
• Ability to operate independently in ambiguous environments and drive technical direction for highly strategic initiatives.
• Strong cross-functional collaboration skills and experience working across engineering, infrastructure, and platform organizations.

Benefits

• Comprehensive healthcare benefits and income replacement programs.
• 401(k) with employer match.
• Global benefit programs covering workspace, professional development, and caregiving support.
• Family planning support.
• Gender-affirming care.
• Mental health and coaching benefits.
• Flexible vacation and paid volunteer time off.
• Generous paid parental leave.

Compensation

The base salary range for this position is $109,800–$325,000 USD. The position is also eligible for equity in the form of restricted stock units and, depending on the position offered, may be eligible to receive a commission.