169K–250K USD / year
Researcher, AI Evaluations
AIRemote — United States
Published on 2026-11-09
These details were extracted automatically from the original listing. They may not be complete or fully up to date — it's worth checking the original job post.
About this role
Lead evaluation research for Figma AI work, defining quality criteria and building studies, benchmarks, and reliable measures that reflect user needs. Partner with AI researchers and engineers to turn findings into model and product improvements and repeatable evaluation practices.
About the company
Figma is a collaborative design platform for product teams to create interfaces, prototypes, and design systems in the browser. It helps designers and developers work together on digital products from ideation through handoff.
The team
AI Insights within Figma’s Research team, working with AI Research within AI Platform.
What you'll do
- Lead evaluation research for priority AI workstreams, from study design and analysis to recommendations
- Develop rubrics, representative datasets, benchmarks, and quality thresholds
- Integrate validated evaluation methods into repeatable evaluations and regression tests
- Communicate quality, failure patterns, and uncertainty to inform model, product, and release decisions
- Validate automated graders, including LLM-as-judge methods, and identify bias and gaps
- Share reusable methods, review work, and mentor peers
What we're looking for
- 7+ years of experience in research, applied research, or a related field
- Track record independently leading complex evaluation projects with measurable impact
- Hands-on experience evaluating AI products and translating findings into model or product improvements
- Expertise in human evaluations, scoring rubrics, datasets, benchmarks, inter-rater agreement, and automated-grader validation
- Strong qualitative and quantitative research skills, including interpreting uncertainty and model behavior
- Strong problem-solving and stakeholder management skills
Nice to have
- Experience integrating automated evaluations and regression testing into engineering workflows; familiarity with Braintrust, LangSmith, or DeepEval
- Experience developing reusable evaluation frameworks and consistent quality standards
- Background in product design, product management, data science, or engineering; familiarity with Figma or similar tools
- Statistical expertise in sampling, significance testing, and agreement metrics
- Advanced training in a quantitative or behavioral discipline
Benefits
- Equity
- Health, dental, and vision coverage
- Retirement benefits with company contributions
- Parental leave and reproductive or family planning support
- Mental health and wellness benefits
- Paid flexible PTO and paid sick leave
- Company recharge days and cell phone and home internet reimbursements