Skip to content
AI Engineer Jobs
Xperteez Technology

AI Evaluation Specialist, Remote - Contract

Xperteez Technology

Location
Remote (United States · United States · United States · United States · United States · +1 more)
Compensation
$30 - $90/hr
Employment
Contract
Level
Entry Level
Posted 4 days ago

About the Role

Xperteez Technology is recruiting an AI Evaluation Specialist to assess AI-generated outputs against quality rubrics, ensuring accuracy and logical consistency. This role involves providing actionable feedback to improve model performance and refining evaluation standards for next-generation AI systems.

Skills

AI Evaluation Quality Assurance Critical Thinking Editorial Review Data Annotation Technical Writing Process Improvement Rubric Development Analytical Skills Attention To Detail Communication Logic Assessment

Perks

  • Remote Work

Full job details

Role Title: AI Evaluation Specialist

Role Type: Contract

Location: Remote (US, CA, UK, IE, AU, NZ)

In this role, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required — your domain knowledge is what matters.

Scope of Work

  • Evaluate AI-generated outputs against detailed rubrics and defined quality standards, focusing on accuracy, relevance, and adherence to guidelines.

  • Apply consistent, impartial judgment across a high volume of examples, ensuring a fair and reliable assessment process.

  • Identify reasoning gaps, tool-use failures, or logic errors in AI assistant responses, providing actionable feedback for iterative improvement.

  • Produce clear, concise written feedback on both strengths and areas for improvement, directly influencing model refinement and AI adoption practices.

  • Participate in discussions regarding rubric interpretation and evolving quality standards, contributing to process optimization and best practices.

  • Maintain meticulous documentation of evaluations and recommendations, ensuring transparency and traceability in assessment workflows.

Preferred Qualifications

  • Experience in grading, quality assurance, editorial review, assessment, annotation, or similar fields demanding careful analysis and detailed feedback.

  • Advanced, daily use of AI assistants (such as ChatGPT, Claude, or similar) as an essential work and productivity tool.

  • Demonstrated ability to synthesize complex information and communicate findings effectively in writing.

  • Background in process improvement, rubric development, or operational quality assessment in an enterprise or educational context.

  • Strong critical thinking skills with a focus on consistency, integrity, and fairness in evaluations.

  • Comfort working independently on large volumes of similar examples while maintaining high attention to detail.

  • Collaborative mindset for sharing insights, discussing ambiguous cases, and refining evaluation criteria as models evolve.