The AI job market, made clearer

Find work worth
your expertise.

Added 17 Oct 2026New on Thriveth

Software Engineer – AI Code Evaluation & Benchmarking (US candidates only)

We are looking for experienced Software Engineers to help evaluate, benchmark, and improve the coding capabilities of frontier AI models.

United States

AI Evaluation & RLHF Coding

Hiring companyTuring

Application process2.0 / 5Based on 1 review

Work experienceN/ANo approved reviews yet

About the role

In this role, you will assess AI-generated code, validate solutions against real-world software engineering tasks, identify correctness and quality issues, and contribute to the development of high-quality evaluation datasets and benchmarks.

This position is ideal for engineers who enjoy code review, debugging, problem-solving, and applying strong software engineering judgment to complex technical scenarios. Your work will directly contribute to measuring and improving the performance of advanced AI coding systems.

Perks of Freelancing With Turing

Work in a fully remote environment.
Opportunity to work on cutting-edge AI projects with leading LLM companies.

Duration of contract : 1 month; [expected start date is next week].

Responsibilities

Review and evaluate AI-generated code for correctness, efficiency, maintainability, and adherence to requirements.
Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
Debug code, reproduce issues, and verify fixes across different programming environments.
Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks.
Identify edge cases, failure modes, and areas where AI systems struggle with software engineering problems.
Document findings clearly and provide structured feedback to improve evaluation quality and consistency.
Collaborate with project teams to establish quality standards and evaluation methodologies.

Required skills

Strong proficiency in one or more of the following languages: Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL.
Strong understanding of data structures, algorithms, software design principles, and debugging methodologies.
Experience performing code reviews and evaluating code quality in production or large-scale codebases.
Ability to analyze complex technical problems and assess solution correctness with minimal supervision.
Familiarity with version control systems (e.g., Git) and modern software development workflows.
Strong written communication skills and attention to detail.

Preferred skills

Experience with AI/ML data annotation, NLP, prompt engineering, model evaluation, or LLM-related projects is a plus.
Experience evaluating AI-generated code, benchmark creation, or software quality assessment is highly preferred.

Experience and education

Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical field.
3+ years of professional software engineering experience.

Assessment requirements

Online automated coding challenge for Python and Docker test (RHLF).

Schedule details

Commitments Required: At least 4 hours per day and minimum 20 hours per week with overlap of 4 hours with PST.

Eligibility

Work arrangement: Fully remote

Eligible countries: United States

Compensation details

Engagement type : Contractor assignment (no medical/paid leave).

More about the hiring companyTuring

Ready to apply?

Apply with Thriveth referral

Referral disclosure

Thriveth may earn a referral payment if you use the referral link and meet the company’s conditions. Your choice does not improve your chances or change how this job is shown.

View referral details

Thriveth may earn a referral bonus if you use this link and meet Turing’s referral conditions. Using the referral link is optional and does not improve your selection chances or affect how we present the listing. Rewards vary by role. Turing determines whether a referral qualifies. Generally, the candidate must be new to Turing, the referral must be credited to the first referrer through Turing’s portal or public referral page, and the candidate must be hired, complete onboarding, and meet the required work milestone—normally at least 10 worked hours. Candidates already in Turing’s applicant system, already under consideration or otherwise known to Turing, current or former employees or contractors, and candidates who applied directly within the previous 90 days do not qualify under the reviewed terms. Referrers must obtain consent before submitting a candidate’s personal information. Rewards remain subject to Turing’s current terms, eligibility checks, and program changes; payment is not guaranteed. Thriveth is independent of Turing. Using it does not affect job order, ratings, or reviews.

Apply without referral

Find AI-training work. Know what to expect.

Thriveth makes AI data-training work easier to find, understand, and navigate. We replace uncertainty with clear opportunities, realistic expectations, and insights from real application journeys.

Find work worthyour expertise.