Senior QA Engineer (AI Testing)
Arthur J. Gallagher and Co.
- Location
- Hybrid (Colombo, UNAVAILABLE)
- Employment
- Full-time
- Level
- Senior Level
About the role
Gallagher is a global insurance brokerage and risk management firm. This role ensures the quality, safety, and reliability of AI-enabled solutions through comprehensive testing strategies, evaluation of AI-specific risks, and production monitoring.
Skills
Perks
- Hybrid Work
Full job details
Overview
The Senior QA Engineer (AI Testing) ensures the quality and reliability of AI-enabled solutions across their lifecycle, including testing, evaluation, and production monitoring. The role involves collaboration with stakeholders to define acceptance criteria, assess risks, and implement testing strategies for AI solutions such as generative AI, machine learning, and intelligent automation. Based in Colombo, Sri Lanka, this position reports to the QA Practice Lead.
How you'll make an impact
- Develop and maintain comprehensive test strategies for AI-enabled solutions.
- Evaluate AI behaviour for accuracy, robustness, fairness, and safety.
- Test AI-specific risks, including hallucination, data leakage, and model drift.
- Build automated regression suites and integrate them into CI/CD pipelines.
- Collaborate with stakeholders to clarify requirements and communicate quality risks.
- Monitor AI quality in production and contribute to continuous improvement.
- Stay updated on industry trends and introduce advanced testing tools and techniques.
About you
- Proven experience in software testing, including AI-specific testing.
- Expertise in AI risks, retrieval-augmented generation, and prompt engineering.
- Familiarity with secure software engineering practices and Agile methodologies.
- Experience with performance, accessibility, and security testing.
- Knowledge of CI platforms (e.g., Azure DevOps) and tools like Postman and JMeter.
- ISTQB certification and training in AI or machine learning are advantageous.
This role is ideal for candidates passionate about advancing AI testing practices in a collaborative environment.