QA Automation Engineer

Own and scale automated testing across our production AI products, from UI and API suites to validating AI agent behaviour.

All open roles

QA

QA Automation Engineer On-site, Bengaluru Full-time 2 to 5 yearsApply on the form

About the role

We are looking for a QA Automation Engineer who can own and scale our quality engineering practices across RevRag AI's production AI products. You will work closely with engineering, product, and AI teams to build robust automated testing frameworks, identify edge cases, and ensure reliable product releases.

This is a hands-on role where you will be involved across the entire testing lifecycle, from test strategy and automation to debugging, regression, and release validation.

Key Responsibilities

• Design, build, and maintain scalable automated test frameworks for web applications, mobile applications, APIs, and backend services.
• Develop and maintain UI, API, integration, regression, and end-to-end automated test suites.
• Identify critical user journeys and convert them into reliable automated test cases.
• Create comprehensive test strategies, test plans, test cases, and automation coverage for new and existing features.
• Perform functional, integration, regression, smoke, sanity, and exploratory testing.
• Work closely with developers and product teams to identify, reproduce, debug, and resolve defects.
• Integrate automated tests into CI/CD pipelines and ensure tests run reliably as part of the release process.
• Analyze test failures, identify root causes, and distinguish between product defects, test issues, and environment issues.
• Build effective test data, mocks, stubs, and test environments where required.
• Establish and track key QA metrics such as automation coverage, defect leakage, failure rates, and release quality.
• Write clear, reproducible bug reports and manage them through a defect-tracking tool (Jira or similar).
• Continuously improve the automation framework, testing processes, and overall quality engineering practices.

AI and Product-Specific Responsibilities

• Test AI-powered products and workflows, including AI agents and conversational systems.
• Validate AI behaviour across different scenarios, inputs, edge cases, and failure conditions.
• Build automated validation for AI responses, conversation flows, API behaviour, and system integrations.
• Test areas such as prompt behaviour, response consistency, intent handling, fallback mechanisms, latency, and failure recovery.
• Work with AI/ML teams to identify quality gaps and develop repeatable approaches for testing AI systems.
• Experience testing Voice AI or Conversational AI products will be highly valuable.

Required Qualifications

• 2-5 years of experience in Software Testing / QA Automation.
• Strong hands-on experience with test automation frameworks.
• Strong programming/scripting skills in Python, Java, JavaScript, or TypeScript.
• Experience with UI automation tools such as Playwright, Cypress, Selenium, or similar.
• Strong experience with API testing and automation using tools/frameworks such as Postman, REST Assured, Pytest, or equivalent.
• Good understanding of HTTP, REST APIs, JSON, databases, SQL, and backend systems.
• Experience with Git, CI/CD pipelines, and debugging test failures.
• Strong understanding of QA methodologies, defect lifecycle, test design techniques, and software development lifecycle.
• Ability to independently own testing for a feature/product area and drive quality improvements.

Good to Have

• Experience working on AI/ML, LLM, Voice AI, Conversational AI, or GenAI products.
• Experience testing LLM/AI agents, prompts, RAG systems, or model-driven workflows.
• Experience with performance, load, or reliability testing.
• Familiarity with cloud platforms such as AWS, GCP, or Azure.
• Experience with Docker and modern DevOps workflows.
• Experience building QA automation frameworks from scratch.
• Familiarity with observability, logging, and monitoring tools.
• Familiarity with test management/tracking tools (TestRail, Zephyr, Jira, or similar).

What We're Looking For

You're a great fit if you:

• Have strong ownership and attention to detail.
• Think beyond happy-path testing and actively look for edge cases and failure scenarios.
• Have strong debugging and problem-solving skills.
• Can work effectively in a fast-paced startup environment.
• Have a quality-first mindset with the ability to balance speed and reliability.
• Have strong communication and collaboration skills.

Success in This Role

You'll be successful if you:

• Stand up a stable automation framework covering the most critical user journeys and APIs within your first few months.
• Meaningfully reduce manual regression effort and time-to-release through automation coverage.
• Catch critical defects, including AI/agent behaviour issues, before they reach production.
• Become the go-to owner for release quality and sign-off across one or more product areas.
• Build repeatable, trusted approaches for testing AI and conversational workflows, in partnership with the AI/ML team.

Why Join RevRag AI

• Work on production-grade AI products used by real customers.
• Get exposure to Voice AI, LLMs, AI agents, and modern automation systems.
• Work closely with engineering, product, and AI teams.
• High ownership and direct impact on product quality.
• Opportunity to build and shape the QA automation function as the company scales.

About RevRag AI

RevRag AI is a Bengaluru-based deep-tech startup building production-grade Voice AI and LLM-powered automation for customer-facing workflows. Our multilingual, in-app AI agents plug directly into BFSI journeys, improving digital onboarding, sales, and support while cutting operational cost by up to 70%.

We work with leading BFSI organisations including Mahindra Finance, Motilal Oswal, PayTM Insurance, PhonePe, KreditBee, Razorpay, Udaan Capital, Kissht, PolicyBazaar, InsuranceDekho, Open Capital, and EaseBuzz.

Apply for QA Automation Engineer See other roles