Backend Software Engineer (Evals) – Support Automation Engineering

Details

About the Team

The Support Automation team at OpenAI scales the organization by applying cutting-edge AI models to real-world challenges, automating and enhancing work across the organization. From customer operations to engineering, the team develops an ecosystem of automation products that empower colleagues and drive impact. The team leverages OpenAI technology (public and pre-released) to improve OpenAI, blending rapid prototyping with a focus on long-term quality, reliability, and reusable solutions.

About the role

You will work as a Backend Software Engineer with experience in ML/LLM-heavy domains to design and build evals infrastructure that measures the quality of OpenAI's support automation. This is a highly cross-functional role focused on building robust backend systems and services that underpin how knowledge is created, accessed, and applied across OpenAI. You will work closely with Data Science and Research partners to design and build evals at scale and integrate OpenAI models into high-leverage workflows.

Responsibilities

• Design eval pipelines that are reliable, reproducible, and extendable.
• Build infrastructure for continuous eval monitoring frameworks (regression/drift monitoring, building robust golden datasets) and feedback loops to strengthen support automation.
• Design, build, and maintain backend services and APIs to support intelligent automation and knowledge systems.
• Integrate and structure data across internal platforms, transforming it into formats optimized for downstream systems and AI workflows.
• Collaborate closely with data, research, and engineering teams to integrate OpenAI models into workflows.
• Own the full development lifecycle of new backend systems and internal platform capabilities.
• Build with scale and maintainability in mind while iterating rapidly on new ideas.

Requirements

• 4+ years of backend engineering experience at product-driven companies (excluding internships).
• Proficiency in backend technologies; tech stack mentioned includes Python, FastAPI, and Postgres.
• Experience designing and scaling distributed systems, APIs, or data processing pipelines.
• Experience building AI agents or applications, including designing evals and improving performance through prompting or scaffolding.
• Familiarity with evaluation methods for LLMs and patterns such as multi-agent workflows, tool use, or long context.
• Experience creating production evals and/or measuring performance of ML/LLM models at scale.
• Pragmatic mindset: comfortable shipping iteratively while building toward a long-term vision.

Benefits & additional info

• Base salary range listed: $255,000 – $405,000 (total compensation also includes equity, potential performance-related bonus(es), and other benefits).
• Medical, dental, and vision insurance with employer contributions to Health Savings Accounts.
• Pre-tax accounts for Health FSA, Dependent Care FSA, and commuter expenses.
• 401(k) retirement plan with employer match.
• Paid parental leave (up to 24 weeks birth parents, 20 weeks non-birthing), paid medical and caregiver leave (up to 8 weeks).
• Paid time off: flexible PTO for exempt employees and up to 15 days annually for non-exempt employees.
• 13+ paid company holidays and multiple coordinated office closures, plus paid sick or safe time as required by law.
• Mental health and wellness support; employer-paid basic life and disability coverage.
• Annual learning and development stipend; daily meals in offices and meal delivery credits as eligible.
• Relocation support for eligible employees and additional taxable fringe benefits (charitable donation matching, wellness stipends).

Other notes

• OpenAI is an equal opportunity employer. Background checks will be administered in accordance with applicable law. Applicants with disabilities can request reasonable accommodations.