Research Scientist – Physicist

Descripción

Summary: Mercor is seeking a Physics AI Evaluator to write prompts, evaluate LLM responses for accuracy, conduct fact-checking, and annotate model performance in physics contexts. Highlights: 1. Refine prompts to guide AI model behavior in physics 2. Evaluate LLM-generated responses for conceptual accuracy and reasoning 3. Conduct fact-checking using authoritative sources and domain knowledge **About the job** **Mercor** connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include **Benchmark**, **General Catalyst**, **Peter Thiel**, **Adam D'Angelo**, **Larry Summers**, and **Jack Dorsey**. **Position:** Physics AI Evaluator **Type:** **Full\-time or Part\-time Contract Work** **Compensation:** **$73/hour** **Location:** **USA, UK, Canada, EU** **Role Responsibilities** * Write and refine prompts to guide model behavior in physics contexts. * Evaluate **LLM\-generated responses** to physics\-related queries for conceptual accuracy and reasoning quality. * Conduct fact\-checking using authoritative public sources and domain knowledge. * Annotate model responses by identifying strengths and areas of improvement. * Assess clarity, structure, and appropriateness of explanations for different audiences. * Apply consistent evaluation standards by following clear taxonomies and benchmarks. **Qualifications** **Must\-Have** * **PhD in Physics or closely related field**. * Deep expertise in **Classical Mechanics \& Dynamics**, **Electromagnetism**, **Thermodynamics**, **Quantum Physics**, or **Relativity**. * Significant experience using **large language models** (LLMs). * **Excellent writing skills** and ability to explain complex physics concepts. * **Strong attention to detail** and experience reviewing technical writing. **Preferred** * Experience with **RLHF**, model evaluation, or data annotation work. * Experience teaching or explaining physics concepts to non\-experts. * Familiarity with evaluation rubrics or structured review frameworks. **Application Process (Takes 20–30 mins to complete)** * Upload resume * AI interview based on your resume * Submit form **Resources \& Support** * For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome * For any help or support, reach out to: support@mercor.com *PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.*