Senior AI Development Lead

at Airbnb
USD 157,000-171,000 per year
SENIOR
✅ Remote

Tech Stack

A/B Testing @ 4 AI @ 7 API @ 4 Communication @ 6 Customer Support @ 4 Experimentation @ 4 LLM @ 4 Machine Learning Python @ 4 SQL @ 7 Statistics @ 7

Details

Airbnb's AI & ML Enablement team in Supply Tech helps business, field, and product teams use AI to automate high-volume processes, launch trustworthy AI features, build labeled datasets, and maintain workflows as models and policies change.

Responsibilities

  • Drive AI workflows and features from design to launch, defining quality standards and measurement plans with engineering, product, and business partners.
  • Prioritize projects by business impact and make trade-offs with managers and business and product leads.
  • Analyze existing processes and determine how AI can automate them while identifying cases requiring human judgment.
  • Design and build AI workflows, including prompts, pipelines, tooling, and quality and drift monitoring.
  • Build evaluation frameworks, test sets, accuracy and agreement metrics, and calibrated LLM judges.
  • Collaborate with engineers on architecture and tooling decisions and evaluate estimates and scope.
  • Translate unclear guidelines into consistent reviewer criteria in collaboration with Product, Legal, and Policy.
  • Create labeling guidelines and produce human-verified labels with subject matter experts.
  • Run labeling programs at scale with vendor teams and assess quality using agreement metrics and spot checks.
  • Define launch criteria for AI features and provide evidence and go/no-go recommendations.
  • Build reusable evaluation and workflow components, document learnings, and help other teams reach production faster.
  • Keep methods current as models, tools, and policies evolve.

Requirements

  • 8+ years of experience in product or program management, analytics, consulting, or applied AI.
  • Proven experience taking technical programs from concept to production and owning outcomes such as backlog, cycle time, or launch speed.
  • Strong SQL skills and knowledge of quality statistics, including accuracy, precision, recall, sampling, and reviewer agreement.
  • Experience shipping an AI workflow, agent, or automation that others rely on, with measurable before-and-after results.
  • Technical fluency sufficient to evaluate engineering estimates and understand prompts, pipelines, and evaluations.
  • Ability to make messy processes measurable and assess success through business outcomes.
  • Strong judgment in evaluating qualitative feedback, reviewer comments, and disagreements.
  • Clear communication skills and the ability to work with sales, operations, product managers, and engineers.
  • Ability to independently investigate problems and build initial solutions.

Nice to Have

  • Hands-on Python and API experience, including scripting data pulls, calling model APIs, or building small automations.
  • Experience with LLM evaluation, test-set creation, LLM judge calibration, or model fine-tuning.
  • Experience with experimentation and measurement, including A/B testing, metric design, and dashboards.
  • Experience in trust and safety, content moderation, customer support automation, annotation operations, or other human-in-the-loop work at scale.
  • Background in a two-sided marketplace, travel, or another operations-heavy business.

Location

This position is US remote eligible. Occasional work at an Airbnb office or attendance at offsites may be required as agreed with the manager. Candidates must live in a state where Airbnb, Inc. has a registered entity.

Compensation

The base pay range is $157,000–$171,000 USD. The role may also be eligible for bonus, equity, benefits, and Employee Travel Credits.

Airbnb is committed to inclusion and belonging, equal employment opportunity, and providing reasonable accommodations throughout the recruitment process.

More jobs at Airbnb

Similar jobs