Scenario Reviewer - AI Trainer
Not Disclosed•ContractorRemote
location_on1536, East 77th Street, Florence-Firestone, Firestone Park, Los Angeles County, California, 90001, United States
About the Company
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
About the Role
As an AI Training Scenarios Reviewer, you will serve as the quality gatekeeper for the personas, digital environments, and tasks created by our design team. Your mission is to ensure that every scenario is logically sound, properly structured, and capable of effectively challenging an AI agent's reasoning, filtering, and prioritization abilities without introducing technical errors.
In this role, you will apply strict rubrics to score scenarios objectively, identifying logical inconsistencies, factual errors, and technical flaws such as invalid JSON structures. You will work closely with scenario designers to provide clear, constructive feedback that refines their worlds and improves overall output. Success in this position requires a keen eye for detail and a strong understanding of complex workflows.
Hiring Process
The application process takes approximately 20–30 minutes to complete and consists of the following steps:
- Upload your resume.
- Complete an AI interview based on your resume.
- Submit the final application form.
Our team reviews applications daily. To be considered for this opportunity, please ensure you complete all AI interview and application steps.
For detailed information regarding the interview process and platform specifics, please visit: talent.docs.mercor.com
For any questions or support, reach out to: support@mercor.com
Work location
Work model: Remote
1536, East 77th Street, Florence-Firestone, Firestone Park, Los Angeles County, California, 90001, United States
Firestone Park, California
Key Responsibilities
- check_circleIdentify logical inconsistencies, factual errors, and technical flaws such as invalid JSON structures
- check_circleProvide clear, constructive, and actionable feedback to scenario designers to refine output
- check_circleDocument review outcomes, scores, and feedback in Airtable and Crucible
- check_circleApply strict rubrics to score scenarios objectively and ensure they challenge AI reasoning
- check_circleAudit and evaluate AI training scenarios for logical soundness and proper structure
Requirements
- verifiedUndergrad degree
- verified2+ years professional experience
- verifiedEditing / QA background
- verifiedJSON familiarity
- verifiedJira/Asana power user
- verifiedSQL/Python exposure
Nice to Have
Editing or QA background (e.g., senior editor, QA tester, technical reviewer, compliance auditor, or peer-reviewer). Technical or data comfort including JSON familiarity, Jira/Asana power user experience, data analyst skills, or SQL/Python exposure. Process and systems thinking experience (e.g., operations managers, technical PMs, or systems designers with a strong understanding of complex workflows).
Similar Job Opportunities
Frequently asked questions about Scenario Reviewer - AI Trainer at Mercor
What does a Scenario Reviewer - AI Trainer at Mercor do?expand_more
In this Scenario Reviewer - AI Trainer at Mercor role, you will identify logical inconsistencies, factual errors, and technical flaws such as invalid json structures; provide clear, constructive, and actionable feedback to scenario designers to refine output; document review outcomes, scores, and feedback in airtable and crucible; and apply strict rubrics to score scenarios objectively and ensure they challenge ai reasoning.
What are the requirements for this Scenario Reviewer - AI Trainer role?expand_more
Mercor is looking for candidates who meet the following requirements: Undergrad degree; 2+ years professional experience; Editing / QA background; JSON familiarity; Jira/Asana power user; and SQL/Python exposure.
Where is the Scenario Reviewer - AI Trainer role at Mercor located?expand_more
Scenario Reviewer - AI Trainer at Mercor is based in 1536, East 77th Street, Florence-Firestone, Firestone Park, Los Angeles County, California, 90001, United States. This is a remote role.
Is this Scenario Reviewer - AI Trainer job remote, hybrid, or on-site?expand_more
Mercor has listed this Scenario Reviewer - AI Trainer role as remote.
How much experience is required for this Scenario Reviewer - AI Trainer role?expand_more
Scenario Reviewer - AI Trainer at Mercor typically requires 2+ years of relevant experience at the mid level level.
What skills do you need for the Scenario Reviewer - AI Trainer role at Mercor?expand_more
Key skills for Scenario Reviewer - AI Trainer at Mercor include Json; Jira; Asana; SQL; Python; Airtable; and Crucible.
What category does the Scenario Reviewer - AI Trainer role belong to?expand_more
Scenario Reviewer - AI Trainer at Mercor is part of the it job category on Recrutus.