Quality & Evaluation Manager

Job Information

Core Skills

AI Testing Areas

• LLM & Generative AI Evaluation
• Reviewer Consistency & Calibration
• Quality Audit Frameworks
• AI Reasoning Evaluation
• Model Training Data Quality
• Red Teaming & Safety Evaluation
• Evaluation Rubrics & Governance
• Operational Quality Analytics

Key Responsibilities

• Lead AI quality operations across Generative AI projects.
• Manage reviewer, auditor, and evaluation teams.
• Design quality audit frameworks and evaluation standards.
• Conduct calibration sessions and improve reviewer consistency.
• Analyze quality metrics and identify systemic risks.
• Collaborate with clients and internal AI teams to improve model quality.

Required Skills

• 5–8 years in Quality Operations or QA leadership.
• 2–3 years working with AI, ML, or Generative AI workflows.
• Experience in audits, calibration, and quality reporting.
• Knowledge of LLM evaluation and AI quality challenges.
• Strong analytical, communication, and stakeholder management skills.
• Ability to work effectively in remote and hybrid environments.

Preferred Skills

• Experience leading global AI data annotation projects.
• Knowledge of reasoning, classification, multimodal AI, and red-teaming.
• Experience with annotation platforms and quality dashboards.
• Background in Linguistics, Philosophy, Social Sciences, or AI-related fields.

Why This Role Matters

High-quality AI models depend on accurate evaluations, consistent reviewer decisions, and reliable training data. This leadership role ensures Generative AI systems meet quality, safety, and performance standards while helping organizations build trustworthy AI solutions at scale.

Disclaimer: This job information is shared for career guidance purposes only. Applications are processed through the employer's official recruitment platform.