# QA Lead - Manual, Automation & AI Testing **Company:** [Apna](http://jobs.workable.com/companies/oGcPwdAqKGbezdH7GyyVKX.md) **Location:** Bengaluru, India **Workplace:** on site **Department:** Engineering [Apply for this job](http://jobs.workable.com/view/73c30748-81bb-40d3-8007-dff9224c6eaa) ## Description **About the role:** We are looking for an experienced and hands-on **QA Lead** with **7+ years of experience** in software quality assurance. The ideal candidate must have strong expertise in **manual testing, automation testing, Python, test automation frameworks, and AI-powered product testing.** The candidate should have experience working in a product-based technology company and be capable of owning the complete quality lifecycle—from requirement analysis and test planning to automation, release sign-off, AI evaluation, and production-quality monitoring. **Role:** QA Lead **Requirement:** 1 **Location:** Bangalore (Domlur | WFO 5 days) **Experience:** 7+ years ## Requirements **Responsibilities:** - Own the overall quality strategy for the assigned products and engineering teams. - Lead manual and automation testing across web applications, mobile applications, APIs, backend services, AI features, and third-party integrations. - Design, develop, and maintain scalable automation frameworks using Python. - Create comprehensive test plans, test scenarios, test cases, and release-quality reports. - Perform functional, regression, integration, API, database, exploratory, and performance testing. - Define testing strategies for AI/ML and Generative AI features, including chatbots, recommendation systems, search, summarisation, classification, and content-generation workflows. - Validate AI-generated responses for accuracy, relevance, consistency, completeness, safety, and business-rule compliance. - Test AI systems for hallucinations, inappropriate responses, prompt injection, data leakage, bias, and edge cases. - Build automated evaluation frameworks and datasets for testing LLM and AI-powered features. - Test Retrieval-Augmented Generation (RAG) workflows, including document retrieval, context relevance, response grounding, and citation accuracy. - Validate AI model and third-party LLM API integrations for reliability, latency, error handling, rate limits, token usage, and cost. - Establish baseline quality metrics and regression suites for AI-generated outputs. - Review product requirements, prompts, workflows, and technical designs to identify gaps and risks early in the development lifecycle. - Define and track quality metrics such as defect leakage, automation coverage, regression effectiveness, release readiness, AI response accuracy, hallucination rate, and latency. - Work closely with Product Managers, Developers, DevOps, Data Scientists, and AI/ML Engineers. - Lead release validation, QA sign-off, production sanity testing, and post-release monitoring. - Analyse production defects, support root-cause analysis, and implement preventive measures. - Mentor QA engineers and promote a strong quality-first culture across Product and Engineering teams. ### Must-Have Qualifications - **6+ years of experience** in software testing and quality assurance. - Strong hands-on expertise in both **manual and automation testing**. - Proficiency in **Python** for developing automation frameworks and test utilities. - Strong experience with tools and frameworks such as **Pytest, Selenium, Playwright, Appium, or Robot Framework.** - Experience in API testing using Postman, Python Requests, REST Assured, or similar tools. - Good knowledge of database testing and strong proficiency in SQL. - Strong understanding of testing methodologies, QA processes, SDLC, and STLC. - Experience with functional, integration, regression, system, exploratory, and end-to-end testing. - Experience integrating automated tests with CI/CD pipelines. - Hands-on experience with Git, Jenkins, GitHub Actions, Jira, or similar tools. - Experience working in a **product-based company** and testing customer-facing products at scale. - Understanding of AI/ML concepts and experience testing AI-powered or Generative AI features. - Understanding of LLM behaviour, including non-deterministic outputs, hallucinations, context limitations, and prompt sensitivity. - Ability to design test datasets, evaluation criteria, and quality metrics for AI-generated outputs. - Strong analytical, debugging, problem-solving, and risk-identification skills. - Good communication, stakeholder-management, and team-leadership capabilities. - Ability to take complete ownership of product quality and release sign-off. ### Good to Have - Experience testing **LLM-based applications**, **AI chatbots, RAG systems, recommendation engines, or semantic search**. - Experience with AI evaluation and observability tools such as LangSmith, DeepEval, Ragas, Promptfoo, TruLens, or similar platforms. - Familiarity with models and APIs from OpenAI, Gemini, Claude, or open-source LLM platforms. - Knowledge of prompt engineering and automated prompt-regression testing. - Experience evaluating AI systems for responsible AI, privacy, security, fairness, and bias. - Experience with performance-testing tools such as JMeter, Locust, or k6. - Experience testing microservices, distributed systems, and event-driven architectures. - Exposure to cloud platforms such as GCP, AWS, or Azure. - Knowledge of Docker, Kubernetes, Kafka, or similar technologies. - Experience with monitoring tools such as Grafana, Kibana, or Datadog. - Experience in recruitment technology, marketplaces, SaaS, or other high-scale products. ### Education Bachelor’s degree in Computer Science, Engineering, Information Technology, or a related field. ### Ideal Candidate The ideal candidate is a hands-on QA leader who combines strong technical expertise with product and AI-quality thinking. They should be comfortable writing automation code, performing detailed manual testing, evaluating AI-generated responses, challenging requirements, identifying customer-impacting risks, and guiding teams towards reliable, safe, scalable, and high-quality product delivery.