Job

ML Inference Engineer – Terminal-Bench Task Designer (vLLM / SGLang / TensorRT-LLM)

Toptal Global freelance talent marketplace connecting clients with remote experts

Remote ML inference and serving expert sought to design terminal-based benchmark tasks for AI agents, with task-based compensation.

Open to Rwandans

At a glance

Location
Remote
Paid
Yes
Minimum years of experience
3
Listed
Source
Remote OK

About this opportunity

Toptal is building a benchmark that evaluates how AI agents handle real expert work in a terminal. They hire ML inference and serving experts to translate real paid tasks into expert-level Terminal-Bench challenges. Each task provides an environment, a working solution, and automated checks to prove correctness.

Tasks are designed to be reproducible and solvable from the command line, often involving model serving with streaming. The compensation is task-based.

The role emphasizes creating end-to-end tasks that are fair, gradable, and based on real-world ML inference experience. You’ll write reference solutions, implement automated checks, and package tasks with Docker and Git for reproducibility.

Responsibilities

  • Design expert-level Terminal-Bench tasks derived from real ML inference work.
  • Define environments, systems, and constraints for each task (e.g., model serving with streaming).
  • Create working reference solutions that run end-to-end within 60–75 minutes of effort.
  • Build automated checks to verify results, including edge cases like numerical drift or metric bugs.
  • Package tasks with Docker and Git; document tools, data, and rules for reviewer validation.
  • Iterate tasks based on reviewer feedback to maintain expert-level clarity and fairness.

Requirements

  • 3+ years hands-on ML inference or serving work (or PhD with active research in the field).
  • Recent hands-on experience in the last 3 years with relevant systems, tools, data, and rules.
  • Experience training, serving, or evaluating models at real scale (multi-GPU or production serving).
  • Hands-on Python and command-line fluency; familiarity with files, CSV/JSON, APIs or databases.
  • Experience with PyTorch or JAX internals and serving stacks like vLLM, SGLang, or TensorRT-LLM.
  • Ability to explain ML failure modes (numerical drift, metric bugs) and how to detect them.
  • Available to start by early November 2026 and complete ~60–75 minute tasks.

How to apply

  • Mention the word ASTONISHED and the code RMmEwMjpjMjA3OjIyODA6MTI4Mzo6MQ== in your application to show you read the job post completely.

Ready to apply?

Openno deadline
No deadline given. Apply early.
Apply on the official site

WorkInRwanda never asks for money to apply. If anyone does, report it. Report this listing

About Toptal

Toptal is a marketplace that connects freelance talent with clients across tech and business. It operates globally and supports remote work, matching experts in software development, design, and other professional services with opportunities from various industries.

See everything from Toptal
No deadline Apply