LLM Red Team Specialist - AI Job at Mercor, New York, NY

  • Mercor
  • New York, NY

Job Description

Job Description

Job Description

About the job

Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark , General Catalyst , Peter Thiel , Adam D'Angelo , Larry Summers , and Jack Dorsey .

Position: LLM Red Team Specialist — Failure Modes & Edge Cases
Type: Contract
Compensation: $60–$90/hour
Location: Remote
Commitment: 35 hours/week

Role Responsibilities

  • Evaluate frontier AI models on coding, ML , and analysis tasks to identify vulnerabilities and failure modes.
  • Design complex tasks that challenge models and are fair for grading.
  • Document findings with clear evidence and reproducible steps.
  • Collaborate with task authors to close loopholes and improve grading.
  • Share insights with researchers to enhance benchmark quality.
  • Work independently and asynchronously to meet deadlines and improve AI model performance .

Qualifications

Must-Have

  • MSc or PhD in a STEM field or equivalent experience.
  • 1+ years in research, research-engineering, security, or AI evaluation.
  • Experience identifying vulnerabilities in LLMs or ML systems .
  • Proficiency in Python and Git .
  • Familiarity with LLM capabilities and evaluation techniques.
  • Ability to work 35 hours/week .

Preferred

  • Experience in AI training , model evaluation, or benchmark/task authoring.

Resources & Support

  • For details about the interview process and platform information, please check:
  • For any help or support, reach out to: support@mercor.com

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.

Job Tags

Contract work, Summer work, Remote work

Similar Jobs

Platinum Coastal Group

Marketing Associate Job at Platinum Coastal Group

 ...ROLE SNAPSHOTThis is an entry-level, on-site position that supports live marketing initiatives connected to sports, entertainment, events, and audience engagement. The role is designed for a people-oriented candidate who enjoys working in a collaborative environment... 

Oceaneering International

GRC Analyst Job at Oceaneering International

As a Sr. GRC Analyst for Oceaneering, you will support the organization's cybersecurity governance, risk management, compliance, and security assurance programs. You will partners with business units, IT, OT, legal, and regulatory stakeholders to implement security controls... 

MIENTKIEWICZEATON LLC

Grist House Craft Brewery: Kitcheneer Job at MIENTKIEWICZEATON LLC

 ...Description Job Description Now Hiring: The Kitcheneer Part chef. Part magician. Part smoke-and-fire engineer. Grist House Craft Brewery + Kitchen Oakdale / Collier, PA (Just 20 mins from Downtown) Role: Back-of-House (BOH) Rockstar: The Kitcheneer Pay: Up to $... 

Multi-Craft Contractors, Inc.

Multi-Division Billing Clerk Job at Multi-Craft Contractors, Inc.

 ...requirements. Prepare and maintain files, correspondence, and drawings on current and completed projects. Receive, log and expedite contractor submittals, requests for information, etc. Assist with submission of estimates to customers to ensure they meet company... 

NorthShore University HealthSystem

Genetic Counseling Assistant: Career Growth in Patient Care Job at NorthShore University HealthSystem

Endeavor Health in Evanston, IL, is seeking a Genetics Counselor Assistant in the Department of Medical Genetics for a full-time role. The position...  ...Candidates need a Bachelor's degree; one year of genetic counseling experience is preferred, with EMR experience (Epic) also...