Lead Quality Engineering & AI Evaluation
Lead software quality, AI evaluation, monitoring and reliability across engineering teams while building scalable QA/QC and AI evaluation practices.
Role Overview
Dhruva Space is looking for an experienced quality engineering and AI evaluation leader to own software QA/QC, AI evaluation, production monitoring and reliability. This role will build the quality and evaluation function from the ground up, establish scalable practices, develop AI-agent evaluation frameworks and lead a high-performing quality team.
Key Responsibilities
- Define and own the software QA/QC strategy across engineering teams, including AI-agent-generated code.
- Design and own evaluation frameworks for AI-agent accuracy, reliability, hallucination, drift, performance and cost.
- Establish production monitoring, observability, alerting and escalation processes for AI-agent performance.
- Build, lead and mentor the quality and AI evaluation team, including hiring and talent development.
- Define quality gates, testing standards and release criteria for software and AI-generated code.
- Establish review and testing standards for AI-assisted and AI-generated software development.
- Define and track software quality and AI-agent performance metrics for leadership reporting.
- Investigate systemic quality issues and drive root-cause analysis and corrective actions.
- Create feedback loops connecting evaluation results, production monitoring, agent performance and prompt/model improvements.
- Collaborate with engineering, product, AI/ML and architecture teams to improve software and AI system quality.
- Act as the primary escalation point for critical software quality and AI-agent performance issues.
- Build scalable QA/QC and AI evaluation practices that can support multiple products and AI agents.
- Establish consistent quality standards as the engineering and AI ecosystem scales.
Ideal Candidate
An experienced Quality Engineering or QA leader with strong hands-on expertise in AI/ML and LLM evaluation. The ideal candidate should be capable of building a quality function from the ground up, defining evaluation methodology, establishing production observability and leading teams in complex engineering environments.
Candidate Requirements
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, AI/ML, Data Science or a related field.
- 8–12 years of experience in QA/QC, Quality Engineering, Software Quality, AI/ML evaluation or related roles.
- Proven experience building or leading a QA/QC or Quality Engineering function, preferably from the ground up.
- Strong practical experience designing evaluation frameworks for ML, LLM or AI-agent systems.
- Strong understanding of AI-agent architectures and metrics including accuracy, hallucination, reliability, drift, latency and cost.
- Strong understanding of software testing, automation, CI/CD, release management and quality gates.
- Experience with AI evaluation and observability tools such as DeepEval, LangSmith, Arize Phoenix or equivalent platforms.
- Experience designing production monitoring, alerting and observability for AI/ML systems.
- Proven experience building, mentoring and managing quality or engineering teams.
- Experience in safety-critical, regulated, aerospace, defence or high-compliance software environments is preferred.
Job Information
Ready to Apply?
Explore the complete job description and submit your application through LinkedIn.
Apply Now on LinkedIn