The ideal candidate is comfortable working across large multi-file codebases, reasoning about how a bug or feature ripples across modules, and building reproducible, containerized environments backed by rigorous tests. You should be able to author a problem a strong engineer would recognize as real work, and a verifier that grades a solution the way a careful reviewer would. Experience with Docker and containerized workflows is required. You will also need a substantial private codebase of your own to author tasks against.

• Author long-horizon coding tasks grounded in real engineering problems (bugs, features, regressions, migrations) from substantial codebases.
• Write clear prompts that require genuine investigation and complicated execution by a coding agent.
• Build deterministic test suites and review criteria that verify correctness, catch regressions, and resist shortcut or "reward-hacking" solutions.
• Containerize tasks with Docker so they build and run reproducibly.

Requirements

• Experience architecting or maintaining substantial, complex, and large software projects.
• Experience using coding agents for long tasks.

Preferred qualifications

• Background in code review, test authoring, or maintaining production and/or open-source systems.
• Exposure to AI evaluation, rubric/data grading, or prompt writing.

Access to a large, complex, and private codebase you own and can author tasks against.

One assessment, ongoing matches.

Work arrangement: Fully remote

$85/hr.