Senior Manager - Testing & AI Quality Engineering
Own the test strategy for AI assistants, copilots and agents across Ramco products, and lead the shift to AI-led testing across the quality organisation.
- Location
- Chennai, India
- Experience
- 18+ years
- Job type
- Full-time
- Reports to
- Head – Quality Engineering
What you will lead. Select a card to jump to its details.
About Ramco Systems
Ramco Systems is part of the $1 billion+ diversified Ramco Group, a legacy enterprise with deep roots in industries such as cement, textiles, engineering, software, energy, and biotech. With over 75 years of heritage and innovation, the Ramco Group is a global powerhouse that blends traditional business values with modern technologies.
Ramco Systems is the Group’s technology arm, delivering next-gen enterprise software across Global Payroll, HR, Aviation MRO, ERP, and Logistics. Headquartered in Chennai, India, Ramco serves over 1,000 customers across 35+ countries and is recognized for its AI/ML-based, bot-enabled, and cloud-native solutions.
Our platform is designed for digital operations—trusted by governments, Fortune 500 companies, and high-growth enterprises alike.
If you’re looking to be part of a company that combines innovation, global exposure, and a mission to disrupt legacy systems with intelligent enterprise applications—Ramco is the place to be.
Key responsibilities
Testing AI & agentic software (core focus)8 items
- Own test strategy for copilots, assistants, LLM workflows, RAG pipelines, and multi-step agents across Ramco products.
- Build evaluation-driven quality: golden datasets, eval harnesses, LLM-as-judge with human calibration, and pass thresholds agreed with Product before release.
- Run regression for models and prompts — version pinning, drift detection, re-baselining on model upgrades, and A/B release comparison.
- Test RAG quality (retrieval precision and recall, groundedness, citation correctness, hallucination rate) and agents (tool selection, trajectories, memory, retries, loops, handoffs, MCP and tool contracts).
- Validate blast radius of autonomous action — permission scoping, approval gates, human-in-the-loop, idempotency, and rollback.
- Lead AI red teaming (prompt injection, jailbreaks, data exfiltration, PII and tenant leakage) and AI non-functional quality (latency, token cost, concurrency, fallback).
- Extend quality into production through tracing, online evals, and drift monitoring, feeding failures back into regression.
- Ensure responsible-AI readiness — bias, explainability, and audit evidence against EU AI Act, NIST AI RMF, and ISO/IEC 42001.
AI-led testing transformation5 items
- Define the AI-in-QA strategy across test design, automation, test data, triage, RCA, and release readiness — including where AI is deliberately not used.
- Personally build testing accelerators with Claude Code or comparable agentic tools, and establish agent engineering practices (prompts, context files, skills, sub-agents, MCP integrations to ALM and CI, shared workflow repositories).
- Drive AI-assisted test design with mandatory human review, NL-to-script automation (Playwright, Selenium), self-healing locators, risk-based test selection, synthetic and masked test data, and AI defect intelligence.
- Govern AI-generated artefacts — review gates, traceability, IP hygiene, secure data handling, and human ownership of every released test asset.
- Measure AI impact (adoption, cycle time, coverage, escaped defects, cost per release), retire what shows no value, and reskill 40–50 engineers in prompt, context, and agent engineering.
Quality strategy & organisational leadership4 items
- Set the quality vision and multi-year roadmap with Product, Engineering, Data Science, Support, and Delivery.
- Run Quality Maturity Assessments with the Quality CoE and ensure QMS adherence.
- Balance Dev and QA capacity, and re-baseline QA sizing as AI gains land.
- Own the operating budget, including AI tooling and token spend, with ROI tracking.
Test engineering, automation & non-functional quality4 items
- Manage strategies across functional automation, API, performance, accessibility, and security testing.
- Drive testability and observability, including the instrumentation AI features need to be evaluable.
- Build simulation environments with mocked model endpoints for deterministic CI.
- Run proofs of concept comparing AI testing platforms against internal builds, and plan end-to-end testing across products and partners.
Delivery, metrics & people leadership4 items
- Track productivity, defect leakage, automation coverage, AI adoption and, for AI features, eval pass rates and hallucination trends.
- Report licensing, financials, delivery, and quality to executives, and resolve production issues with global stakeholders.
- Lead managers, leads, and engineers across performance, automation, accessibility, security, and AI quality.
- Define career paths for AI quality and test tooling engineers, and build a culture of experimentation and psychological safety.
What success looks like
- AI features released against defensible, pre-agreed evidence rather than subjective confidence.
- Non-deterministic behaviour caught in evals before it reaches a customer workflow.
- AI used in QA where it demonstrably earns its place, and retired where it does not.
- Escaped defects trending down across products while release cadence goes up.
- A test organisation whose engineers can build their own tooling with coding agents.
- Responsible-AI evidence available when a customer, auditor, or regulator asks for it.
Key performance indicators
Performance in this role is measured against the following indicators. Targets are set annually with the Head of Quality Engineering and reviewed on the cadence noted.
AI release confidence
Eval pass rates against agreed thresholds, hallucination rate, and groundedness and citation correctness on RAG features at release.
Review cadence: Per release
AI safety and blast-radius assurance
Red-team findings (prompt injection, jailbreak, data leakage) closed before release, and permission scoping, approval gates, and rollback validated for autonomous actions.
Review cadence: Per release
AI-in-QA impact
Adoption of AI testing accelerators, cycle-time reduction, coverage gain, and cost per release against baseline.
Review cadence: Quarterly
Escaped defects and production quality
Defect leakage to production across products, and production issues resolved within agreed timelines.
Review cadence: Monthly
Automation and non-functional coverage
Functional automation, API, performance, accessibility, and security test coverage against plan.
Review cadence: Quarterly
Quality maturity and governance
Quality Maturity Assessment outcomes, QMS adherence, and responsible-AI evidence maintained and audit-ready.
Review cadence: Half-yearly
Capability and budget stewardship
Engineers reskilled in prompt, context, and agent engineering against plan; QA sizing re-baselined; and operating budget including AI tooling and token spend managed to plan with ROI tracked.
Review cadence: Half-yearly
Required experience & skills
Education & leadership experience3 items
- Bachelor’s or Master’s degree in Computer Science or Engineering.
- 18+ years in test engineering, including 10+ years as a QA manager on complex products.
- 5+ years leading 50+ engineers including managers, in a matrixed global MNC, with a proven shift from QA to test engineering to AI-augmented testing.
AI & agentic testing — essential5 items
- Hands-on tools built with Claude Code or equivalent, with concrete examples of users and savings; agent engineering across context files, skills, sub-agents, and MCP.
- Production experience testing LLM and agentic software — copilots, RAG, multi-step workflows — not lab-only.
- Eval frameworks for non-deterministic systems: golden datasets, LLM-as-judge with human calibration, and defensible release criteria; RAG quality measurement.
- AI safety testing — prompt injection, jailbreaks, tool data leakage, and agent permission scoping.
- Tooling: LangSmith, Langfuse, Braintrust, DeepEval, Ragas, promptfoo, and agent-run tracing; judgement on AI economics and when deterministic tests are the better answer.
Core test engineering5 items
- Functional and non-functional testing, Agile and Scrum, SQL validation, and quality standards and metrics for strategic programmes.
- Automation and performance tooling: Playwright, Selenium, Cucumber, Postman, JMeter, LoadRunner; REST, SOAP, GraphQL, and event-driven APIs.
- Languages and DevOps: Python, TypeScript/JavaScript, Java or C#; Git, Jenkins, containers, Terraform or Ansible; LLM APIs and orchestration patterns.
- ALM: Jira/Xray, Azure DevOps, TestRail, Zephyr, with APIs for agent-driven integration.
- ISTQB Advanced Test Manager certification.
Preferred qualifications5 items
- AI quality gates embedded in CI/CD.
- Exposure to EU AI Act, NIST AI RMF, and ISO/IEC 42001.
- Human-in-the-loop models for agents operating on enterprise data.
- AI enablement contributions — training, guilds, hackathons.
- AI certification such as CT-AI; PMP or PRINCE2 an advantage.
Why join Ramco Systems
- Lead quality for AI assistants, copilots, and autonomous agents embedded in enterprise workflows used across 35+ countries.
- A genuine dual mandate — testing AI, and rebuilding testing itself around AI.
- Budget and authority to build tooling rather than only buy it.
- Be part of a fast-evolving, innovation-led enterprise software company.
Ready to apply?
See the full job description and apply on LinkedIn.