AI Agent Evaluation Analyst

We're looking for AI Agent Evaluation Analysts to review evaluation tasks and scenarios for logic, completeness, and realism. The project involves identifying inconsistencies, missing assumptions, or unclear decision points, and annotating cause-effect relationships, reasoning paths, and plausible alternatives.

Requirements

• Excellent analytical thinking: Can reason about complex systems, scenarios, and logical implications.
• Strong attention to detail: Can spot contradictions, ambiguities, and vague requirements.
• Familiarity with structured data formats: Can read, not necessarily write JSON/YAML.
• Ability to assess scenarios holistically: What's missing, what’s unrealistic, what might break?

Benefits

• Get paid for your expertise, with rates that can go up to $44/hour depending on your skills, experience, and project needs.
• Take part in a flexible, remote, freelance project that fits around your primary professional or academic commitments.
• Participate in an advanced AI project and gain valuable experience to enhance your portfolio.
• Influence how future AI models understand and communicate in your field of expertise.