Job

Expert Talent Pool, Evals and Benchmarks

Clera Clera offers roles in legal evaluation of AI systems and ML internship opportunities

Senior evaluation role designing benchmarks and leading grading for AI evaluation projects. Fully remote worldwide.

Open to Rwandans

At a glance

Deadline
Location
Remote
Salary
75 – 175 USD / hour
Paid
Yes
Employment type
Full Time
Minimum years of experience
5
Listed

About this opportunity

Join Clera’s network of senior evaluation experts supporting AI research projects. You will design benchmarks, write rubrics, develop gold-standard answers, and lead grading operations to ensure rigorous assessments of AI models.

This fully remote, senior-level position offers project timelines ranging from one week to several months, with compensation of $75 to $175 per hour depending on project and experience.

You’ll calibrate graders, analyze model failures, run model comparison studies, and validate benchmarks for contamination and ambiguity. Strong English communication and Python proficiency are essential.

Projects span worldwide, with flexible scheduling across many time zones. Expect to contribute to multiple evaluations and potentially long-term collaborations.

What you get

  • Compensation of $75 to $175 per hour
  • Fully remote work
  • Project-based engagements from one week to several months
  • Exposure to cutting-edge AI evaluation projects

Responsibilities

  • Design benchmarks: scope, question types, difficulty, and scoring methods
  • Write and review rubrics for consistent grading
  • Develop gold-standard answers and test sets
  • Lead grading rounds: calibrate graders, measure agreement, resolve disputes
  • Analyze model failures and recommend improvements
  • Design and conduct model comparison studies
  • Validate benchmarks for contamination, ambiguity, and saturation
  • Design user studies on how people rate and interact with model outputs

Requirements

  • At least 5 years of experience in evaluation, testing, or assessment
  • Experience designing a benchmark or assessment
  • Experience writing or reviewing rubrics and gold-standard answers
  • Proficiency in Python and evaluation tools (Inspect, lm-eval-harness, HELM, promptfoo)
  • Experience analyzing model/test failures and recommending fixes
  • Background in ML evaluation, QA, psychometrics, or related fields
  • Clear written English communication

How to apply

  • Apply through the Himalayas platform listing linked in the post

Ready to apply?

55days left
Deadline
Apply on the official site

WorkInRwanda never asks for money to apply. If anyone does, report it. Report this listing

About Clera

Clera is involved in legal and artificial intelligence work, offering roles that blend EU law expertise with AI evaluation. The listings indicate remote opportunities available worldwide and focus on senior legal professionals with EU knowledge as well as entry-level ML work.

See everything from Clera
55 days left Apply