ProductJobs.io
← Back to all jobs

Job Description

Build and scale how Expedia Group measures search and recommendations quality across brands. You will define evaluation strategy, partner closely with ML teams to assess ranking and recommendation tradeoffs, and use experimentation and benchmarking to shape a roadmap grounded in measurable traveler outcomes.

What you’ll work on

  • Own the product vision and roadmap for search and recommendations quality measurement, translating target outcomes and KPIs (such as relevance, ranking quality, recommendation engagement, and search-to-booking success) into delivery milestones.
  • Partner with ML Science and ML Engineering to define and evaluate model quality using offline metrics (for example, NDCG@10) and online proxies (for example, Average Booking Position, recommendation click-through, and conversion), enabling technical tradeoff decisions.
  • Own and scale the evaluation system for search relevance, including a curated query corpus, human relevance annotation, and LLM-as-judge scoring. Report results through the broader Search Health program and extend the approach to recommendation surfaces (such as destination recommendations, cross-sell, and rebooking recommendations).
  • Define and maintain product metrics for search relevance and recommendation quality, running regular metric reviews and ensuring the roadmap stays accountable to outcomes across both surfaces.
  • Lead competitive relevance and recommendation benchmarking against key players, translating findings into a prioritized set of investments across product, ranking, and personalization.
  • Coordinate experimentation and win-loss analysis for ranking, relevance, and recommendation changes, recommending strategies to improve quality across brands.
  • Coach and influence PMs and stakeholders across the wider Search & Recs organization on measurement best practices.

Skills and qualifications

  • Bachelor’s degree in a technical, quantitative, or related field (or equivalent practical experience), plus 8+ years of product management experience (or 6+ years with a Master’s).
  • Experience owning product metrics, experimentation, or evaluation frameworks for ranking, recommendations, personalization, or search systems.
  • Strong technical depth, including direct partnership with ML Science and ML Engineering on model evaluation and tradeoffs.
  • Proven ability to design product experiments independently, interpret ranking and relevance (or recommendation engagement) data, and translate insights into a defensible roadmap.
  • Track record of driving milestone and release planning for complex, cross-functional initiatives, with proactive risk mitigation.
  • Strong written and verbal communication, including experience presenting to and influencing senior stakeholders.

Tools and methods

  • NDCG@10
  • LLM-as-judge

Benefits

  • Medical, dental, and vision coverage
  • Paid time off
  • Employee Assistance Program
  • Wellness and travel reimbursement
  • Travel discounts
  • International Airlines Travel Agent Network (IATAN) membership

Preferred qualifications

  • Experience with search ranking and/or recommendation systems (such as learning-to-rank models, collaborative filtering, NDCG, engagement and conversion metrics).
  • Familiarity with LLM-as-judge or other AI-assisted relevance and recommendation evaluation methods.
  • Experience in marketplace or travel/e-commerce search and recommendations, balancing traveler relevance with supplier and commercial considerations.
  • Experience running or contributing to competitive benchmarking programs across search and/or recommendation surfaces.

Compensation: USD 173,000 - 277,000 per yearly. Seattle cash range: $173,000.00 to $242,500.00. Starting pay varies by location, available budget, and individual knowledge, skills, and experience. Pay ranges may be modified in the future.

Accommodation: Expedia Group is committed to an inclusive and accessible recruiting experience. If you need an accommodation or adjustment due to a disability during the application or recruiting process, submit a request at https://expedia.service-now.com/askeg?id=job_accommodation.

Similar Jobs