157K–171K USD / year
Senior AI Development Lead
AIRemote — United States
Published on 2026-11-19
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
Lead the design, evaluation, and launch of AI workflows and features that automate operational tasks or help product teams validate AI features. Partner with business, product, engineering, and subject matter experts to establish quality standards, prove results with data, and scale effective approaches.
About the company
Airbnb is a global online marketplace that connects travelers with hosts offering unique accommodations and experiences in various destinations. The platform serves individuals seeking short-term lodging as well as hosts looking to monetize their properties.
The team
AI & ML Enablement team in Supply Tech
What you'll do
- Drive supported AI workflows and features from design through launch, defining quality standards and measurement plans.
- Map existing processes and design AI workflows, including human review for complex cases.
- Build evaluation frameworks, test sets, quality metrics, and calibrated LLM-judge processes.
- Create labeling guidelines and manage human-verified labeling with subject matter experts and vendors.
- Partner with engineering on architecture, tooling, estimates, and scope.
- Recommend go/no-go decisions for AI launches based on evaluation evidence.
- Develop reusable evaluation and workflow components, document learnings, and help teams adopt them.
- Monitor quality and drift and update methods as models, tools, and policies change.
What we're looking for
- 8+ years in product or program management, analytics, consulting, or applied AI, with a record of taking technical programs to production.
- Strong SQL and knowledge of quality statistics, including accuracy, precision, recall, sampling, and reviewer agreement.
- Experience shipping a relied-upon AI workflow, agent, or automation, with measurable before-and-after results.
- Technical fluency to assess prompts, pipelines, evaluations, engineering estimates, and implementation trade-offs.
- Ability to make messy processes measurable and focus on business outcomes.
- Strong judgment in interpreting qualitative feedback and reviewer disagreements.
- Clear communication and ability to deliver outcomes across teams without direct authority.
- Comfortable taking initiative and prototyping solutions directly.
Nice to have
- Hands-on Python and API experience.
- Experience with LLM evaluation, test sets, LLM-judge calibration, or model fine-tuning.
- Experience with experimentation, A/B testing, metric design, or dashboards.
- Experience in trust and safety, content moderation, customer support automation, annotation operations, or human-in-the-loop work at scale.
- Experience in a two-sided marketplace, travel, or an operations-heavy business.
Benefits
- May be eligible for bonus, equity, benefits, and Employee Travel Credits.
