GenAI Testing Consultant / Architect

Job Information

Core Skills

AI Testing Areas

• LLM Evaluation Frameworks
• Retrieval-Augmented Generation (RAG)
• Prompt Engineering Validation
• Groundedness & Relevance Testing
• Toxicity & Safety Evaluation
• Latency & Cost Optimization
• Model Drift Detection
• Cloud AI Architecture Validation
• Agentic AI Testing
• MCP Workflow Validation

Key Responsibilities

• Design enterprise-grade Generative AI architectures.
• Lead GenAI discovery workshops and solution roadmaps.
• Build scalable AI applications using Python and cloud platforms.
• Validate LLM applications using modern AI evaluation frameworks.
• Measure relevance, groundedness, toxicity, latency, and AI cost metrics.
• Implement MLOps, LLMOps, observability, and AI monitoring.
• Drive AI architecture governance and technical best practices.

Required Skills

• Experience designing enterprise Cloud AI architectures.
• Strong knowledge of RAG, Prompt Engineering, MCP, and Agentic AI.
• Hands-on experience with MLOps and LLMOps.
• Python development for AI solutions.
• Experience with AWS, Azure, or Google Cloud.
• Knowledge of AI evaluation metrics and AI quality engineering.

Preferred Skills

• CI/CD and DevOps practices.
• API Testing and SQL.
• Feature Stores and Data Engineering.
• Batch & Stream Processing.
• Architecture Governance.
• Performance and Cost Optimization.

Available Locations

Bangalore • Pune • Hyderabad • Chennai • Trivandrum • Indore • Nagpur • Mangalore • Mysore • Noida • Bhubaneswar • Kolkata • Coimbatore • Hubli

Why This Role Matters

Modern enterprise AI applications require scalable architecture, reliable evaluation, and continuous monitoring. This role combines AI consulting, cloud architecture, LLM testing, MLOps, and quality engineering to build secure, production-ready Generative AI solutions.

Disclaimer: This job information is shared for career guidance purposes only. Applications are processed through the employer's official recruitment platform.