Evaluation Scientist

Design the evaluation science behind verified expert data: how identity and expertise verification change label quality, and how to prove it with methodology that survives peer review.

What you'll do

• —Lead grader-reliability and expertise-effect studies end to end
• —Develop grading rubrics with domain experts and prove inter-rater reliability
• —Design quality methodology for multilingual and domain-expert pipelines
• —Represent our methodology publicly — in writing and with partners

What we're looking for

• —PhD or equivalent research experience in ML, statistics, or a measurement-heavy field
• —Published work on evaluation, benchmarking, or human judgment
• —Statistical rigor that survives peer review
• —Taste for questions where 'it depends' is the wrong answer