AI Engineer - Quality and Post-Training

Details

Glean is building the next generation of enterprise AI assistants and autonomous agents that reason, plan, and act across complex, multi-step workflows. This role works at the intersection of applied research, quality, and product engineering to improve the reliability and usefulness of AI agents.

The focus is on quality improvements across Glean's models and agent harnesses, including hill-climbing with evaluations, evolving evalsets and agentic verifiers, post-training models for use cases such as model routing, and training task-specific models for enterprise workflows. The role also involves bringing emerging research into the product and working with product teams and customers to turn evaluation and production learnings into quality improvements.

Responsibilities

• Use evaluation results and production traces to improve the agent harness, model routing, prompts, and retrieval while balancing quality, latency, and cost.
• Post-train models using supervised fine-tuning (SFT) and preference- or reward-based optimization, including DPO, PPO, and GRPO.
• Generate data for model training.
• Build reinforcement-learning environments that simulate enterprise workflows, including relevant evalsets, harness support, and LLM-based verifiers for hill-climbing and internal and external benchmarking.
• Improve representative evalsets, LLM judges, and agentic verifiers.
• Calibrate evaluation systems against human graders so scores can be trusted for release decisions and model training.
• Work with product, platform, and research teams to deliver quality improvements in production.

Requirements

• 5+ years of industry experience in AI or machine learning engineering.
• Hands-on experience post-training large language models, including adapting open-weight models to specialized tasks.
• Experience building evaluations for AI assistant and agent systems, including generating evalsets, building LLM-as-a-judge verifiers, and calibrating offline and online signals.
• Good judgment about when a metric reflects genuine quality improvements.
• Strong coding skills in Python and/or Go. TypeScript is a plus.
• Proficiency in an ML framework of choice.
• A track record of shipping production machine learning systems.
• A product-minded approach, with the ability to turn vague problems into concrete experiments and work effectively in a close-knit, cross-functional team.

Location

• Hybrid schedule requiring four days per week in the Mountain View or San Francisco office.

Compensation And Benefits

• Standard base salary range of $200,000–$280,000 annually.
• Compensation may also include variable compensation, equity, and benefits, depending on the role.
• Medical, vision, and dental coverage.
• Generous time-off policy.
• 401(k) plan contributions.
• Home office improvement stipend.
• Annual education and wellness stipends.
• Regular company events and healthy daily lunches.

Glean is committed to building and sustaining a diverse, inclusive workplace. As part of the interview process, candidates complete a brief AI-focused exercise or discussion about how they think about, design, and use AI to drive impact.