GENERATIVE AI TESTING

GenAI Test Lead

Lead AI Quality Engineering & Customer-Facing AI Testing

8+ Years Experience • Python • LLM Testing • RAG • AI Automation • AI Governance

LLMs Python RAG AI QA

Job Overview

This opportunity is ideal for experienced AI Quality Engineers who want to lead end-to-end testing of production-ready Generative AI applications. The role focuses on AI testing strategy, LLM evaluation, prompt validation, RAG testing, AI automation, customer engagement, and AI quality governance while ensuring enterprise-grade AI systems are reliable, secure, and production-ready.

💼 Experience
8+ Years

🤖 Domain
Generative AI Testing

💻 Skills
Python • LLM • Automation

📍 Role
GenAI Test Lead

🚀 Apply on LinkedIn

Core Skills

LLM Testing Prompt Engineering RAG AI Agents Python Pytest Playwright DeepEval LangSmith Promptfoo Jenkins GitHub Actions

Key Responsibilities

Lead complete AI testing lifecycle for GenAI applications.
Define AI testing strategies for LLMs, RAG systems, and AI Agents.
Perform prompt regression, hallucination, bias, fairness, and explainability testing.
Develop Python automation frameworks using Pytest and Playwright.
Automate AI evaluation pipelines with DeepEval, LangSmith, Ragas, and Promptfoo.
Integrate AI testing into CI/CD pipelines.
Lead customer meetings, defect triage, UAT, and release sign-off.
Mentor QA engineers in AI testing best practices.

Required Skills

8+ years in Software Testing & QA.
Experience testing production-grade AI applications.
Hands-on LLM and Prompt Engineering validation.
Python Automation (Pytest, Selenium, Playwright).
API Testing using Requests & Postman.
Cloud Platforms (AWS, Azure, GCP).
Knowledge of AI Governance, ISO/IEC 42001, NIST AI RMF.
Experience working with Agile teams.

Technology Stack

Python
Pytest
Playwright
Selenium
Postman
Requests
LLMs
GPT
Claude
Llama
DeepEval
LangSmith
Ragas
Promptfoo
GitHub Actions
Jenkins
AWS
Azure
Google Cloud

Why Apply?

This role offers the opportunity to lead enterprise-scale AI quality engineering initiatives while working on modern Generative AI applications. You'll own AI testing strategy, collaborate with AI engineers, interact directly with customers, and shape the quality standards for production-ready AI systems using the latest AI evaluation frameworks and automation technologies.

Apply Now →

Disclaimer: This job summary is published for informational purposes only. Please apply through the employer's official recruitment portal using the Apply button above.