In this role, you will analyze structured data, create challenging prompts, evaluate AI-generated responses for factual accuracy and reasoning quality, and provide evidence-backed feedback to improve model performance.

We are looking for curious, detail-oriented professionals who enjoy solving complex problems and evaluating AI systems. The ideal candidate can analyze data, think critically, validate responses against evidence, and clearly articulate why a model's output is correct or incorrect. Experience working with AI models and creating challenging evaluation prompts is a strong advantage.

Employment type: Contractor assignment (no medical/paid leave).

Benefits

• Opportunity to work on cutting-edge AI projects.
• Competitive compensation.
• Flexible working hours and remote work environment.

• Create challenging prompts that evaluate an LLM's ability to retrieve, analyze, and reason over structured data.
• Assess AI-generated responses for factual accuracy, logical reasoning, and completeness.
• Identify model failures, inconsistencies, hallucinations, and reasoning gaps.
• Validate model outputs using provided datasets and supporting evidence.
• Document findings with clear, evidence-based explanations.
• Consistently follow annotation guidelines and maintain high-quality standards.

• Master's degree or higher in any discipline.
• Minimum 3 years of professional, research, or teaching experience.
• Strong analytical and critical thinking skills.
• Exceptional attention to detail and ability to validate information against source data.

Shortlisting based on qualifications and assessment scores.

Work arrangement: Fully remote