LLM Evaluation Product Operations Specialist

Responsibilities

• Lead the end-to-end design and rollout of the Tako AI chatbot evaluation framework, enabling ongoing product iteration and optimization.
• Leverage user research and data analysis to identify user needs, define the ideal experience quality standard, and drive improvements in model capability and user satisfaction.
• Collaborate with strategy and algorithm teams to build a scenario-based evaluation metrics system for Tako; deliver analytical insights and recommendations to improve search-user experience and retention.
• Partner with international delivery teams to execute evaluation programs and apply automated evaluation methods to ensure timely, high-quality delivery.

Requirements

• Bachelor's degree or above.
• Strong written communication skills and the ability to think abstractly; capable of clearly defining diverse user scenarios and articulating experience quality standards.
• Highly detail-oriented and well-organized; strong logical thinking with a solid data mindset.
• Excellent communicator and collaborative team player.
• Prior experience in LLM evaluation or user-facing product roles.

Benefits

• Day one access to medical, dental, and vision insurance.
• 401(k) savings plan with company match.
• Paid parental leave.
• Short-term and long-term disability coverage.
• Life insurance.
• Wellbeing benefits.
• 10 paid holidays per year.
• 10 paid sick days per year.
• 17 days of Paid Personal Time (prorated upon hire with accruals by tenure).
• Potential eligibility for discretionary bonuses/incentives and restricted stock units.

About the Team

TikTok's Tako team is seeking a sharp, organized Product Operations specialist to help design and operationalize the evaluation framework for Tako's in-app AI chatbot. The role bridges user insights, data analysis, and cross-functional execution to drive continuous product improvement.

Location

San Jose, CA (onsite)

Employment Type

Regular

Job Code

A77808

Compensation

The base salary range for this position in San Jose is $93,600 - $220,400 annually. Compensation may vary based on qualifications, skills, experience, and location. This role may be eligible for discretionary bonuses/incentives and restricted stock units. Day-one benefits include medical, dental, and vision insurance, and a 401(k) savings plan with company match.