Gen AI Automation Testing Architect

Job Information

Core Skills

Key Responsibilities

• Design scalable automation frameworks for Generative AI applications.
• Develop testing strategies for Large Language Model (LLM) based systems.
• Validate GPT, Llama, Mistral, and other AI models for enterprise use cases.
• Collaborate with AI/ML teams to improve model quality and reliability.
• Integrate AI testing into DevSecOps, CI/CD, and MLOps workflows.
• Manage AI test data, automation pipelines, and quality engineering practices.
• Drive enterprise AI testing standards, performance optimization, and scalability.

Required Skills

• 8–12 years of QA and Automation Testing experience.
• 3+ years of hands-on Generative AI experience.
• Strong Python programming and automation expertise.
• Experience with LangChain, LlamaIndex, Hugging Face, and Vector Databases.
• Knowledge of Azure OpenAI, AWS Bedrock, and Google Gemini.
• Hands-on Docker, Kubernetes, Terraform, DevSecOps, and CI/CD pipelines.
• Strong understanding of AI model evaluation and prompt engineering.

Technology Stack

• Python
• LangChain
• LlamaIndex
• Hugging Face
• GPT
• Llama
• Mistral
• Azure OpenAI
• AWS Bedrock
• Google Gemini
• Docker
• Kubernetes
• Terraform
• CI/CD
• DevSecOps
• Vector Databases

Why This Role Matters

Enterprise AI solutions require robust quality engineering to ensure accuracy, scalability, security, and reliability. This role focuses on building next-generation AI automation frameworks, validating LLM applications, integrating AI testing with DevSecOps and MLOps, and delivering enterprise-grade Generative AI solutions through modern quality engineering practices.

Disclaimer: This job information is shared for informational purposes only. Applications are processed through the employer's official recruitment platform.