Tech Stack
Tag name is followed by "@" symbol and proficiency level value.
About proficiency levels:
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
AI @ 5
Experimentation @ 3
Machine Learning @ 3
Observability
Statistics @ 6
- 1-2 — basic awareness. Minimal hands-on experience, and a rudimentary understanding of the technology's purpose;
- 3-6 — daily use. Comfortable and regular usage, capable of handling common tasks and challenges related to the technology;
- 7-9 — you are an expert, you can teach others, you know all the pitfalls and tricks;
- 10 — exceptional knowledge, comprehensive understanding, and adeptness in all aspects of the technology, including advanced problem-solving. Think twice before claiming or demanding such level.
Details
The Applied AI team builds classification and evaluation systems that provide post-training and research teams with fast, reliable feedback on model behavior. The work combines machine learning and production engineering to develop accurate, efficient, and scalable capabilities.
The role focuses on developing classification and evaluation capabilities and bringing them into production. Responsibilities span supervised fine-tuning, distillation, model evaluation, deployment, real-time inference, and ongoing operation of production classifiers.
Responsibilities
- Build, deploy, and operate production classification and evaluation systems.
- Optimize production inference, including model serving, batching, caching, throughput, latency, compute utilization, and cost.
- Train and optimize classifiers using supervised fine-tuning and distillation.
- Validate model quality and efficiency before deployment.
- Own the reliability and observability of deployed classifiers, diagnose production issues, and improve performance over time.
- Design rigorous evaluations covering accuracy, calibration, generalization, and potential measurement bias.
- Partner with software engineers to integrate models into reliable execution systems and support safe rollouts.
- Build reusable engineering tools for classifier training, evaluation, deployment, and iteration.
- Work with post-training researchers and data scientists to translate their needs into modeling objectives, evaluation criteria, and production requirements.
Requirements
- Experience taking machine learning projects from problem formulation and experimentation through production deployment.
- Strong foundations in supervised learning, statistics, and experimental design.
- Practical experience evaluating models, investigating systematic errors, and working with imperfect labels.
- Strong software engineering skills and the ability to build maintainable production machine learning systems.
- Experience reasoning about inference performance and tradeoffs between quality, latency, and compute cost.
- Ability to combine careful scientific judgment with a practical approach to shipping and operating software.
- Comfort working closely with researchers and engineers on open-ended problems.
- Fluency in using AI agents to move quickly across multiple workstreams in a fast-paced environment.
- Experience with model distillation, representation learning, calibration, or efficient inference is a plus.
Benefits
- Medical, dental, and vision insurance, with employer contributions to Health Savings Accounts.
- Pre-tax accounts for health and dependent care expenses, as well as commuter expenses.
- 401(k) retirement plan with employer match.
- Paid parental, medical, and caregiver leave.
- Paid time off, paid company holidays, office closures, and paid sick or safe time as required by law.
- Mental health and wellness support.
- Employer-paid basic life and disability coverage.
- Annual learning and development stipend.
- Daily office meals and eligible meal delivery credits.
- Relocation support for eligible employees.
- Additional benefits may include charitable donation matching and wellness stipends.
OpenAI is an equal opportunity employer committed to reasonable accommodations and fair employment practices.
More jobs at OpenAI
Strategic Delivery Lead, Critical Infrastructure
OpenAI · Washington, United States
USD 266,000-295,000 per year
Software Engineer, Cloud Computer
OpenAI · San Francisco, United States
USD 255,000-445,000 per year
Data Program Manager, Science
OpenAI · San Francisco, United States
USD 207,000-230,000 per year
Machine Learning Engineer, Growth
OpenAI · San Francisco, United States
USD 381,000-423,000 per year
Researcher, Safety - Frontier Assurance
OpenAI · San Francisco, United States
USD 380,000-500,000 per year
Similar jobs
Software Engineer, Implicit Signals
OpenAI · San Francisco, United States
USD 347,000-445,000 per year
Senior Data Scientist, Ads Integrity
Reddit · United States
USD 190,800-267,100 per year
Research Engineer, Codex
OpenAI · San Francisco, United States
USD 295,000-445,000 per year
Machine Learning Engineer, Growth Platform
Stripe · South San Francisco, United States, New York City, United States, Seattle, United States, United States, Chicago, United States
USD 180,000-270,000 per year
Staff Machine Learning Engineer, Retrieval
Reddit · New York City, United States, United States, San Francisco, United States, Los Angeles, United States, Chicago, United States
USD 230,000-322,000 per year
Member of Technical Staff (Software Engineer, Data Platform)
Perplexity AI · Palo Alto, United States, San Francisco, United States, New York City, United States
USD 220,000-405,000 per year
Senior/Staff Applied Scientist
Glean · Mountain View, United States, San Francisco, United States
USD 180,000-330,000 per year
Data Scientist
Stripe · New York City, United States, Seattle, United States
USD 159,000-288,000 per year