QA Engineer – AI Systems
ConfidencEway
- Location
- Remote
- Employment
- Full-time
- Level
- Entry Level
About the role
ConfidencEway is seeking a QA Engineer to ensure the quality and reliability of its AI-powered products, including AI Assistants and agent systems. The role focuses on evaluating AI accuracy, managing test coverage, and collaborating with engineering teams to maintain high standards.
Skills
Perks
- Remote Work
Full job details
This is a remote position.
Key Responsibilities
-
Evaluate AI agents for accuracy, consistency, and appropriate handling of edge cases and failures
-
Assess multi-step, multi-tool AI workflows to confirm agents behave as intended across an entire task, not just a single response
-
Conduct functional, regression, integration, and user acceptance testing across web and API layers
-
Identify, document, troubleshoot, finding root cause and track defects, working closely with the development team through to resolution
-
Maintain clear and thorough test documentation for both conventional features and AI-driven functionality
-
Support automation testing initiatives where relevant, or help establish them if they do not yet exist
-
Work with product and engineering teams to translate requirements into meaningful test coverage
Requirements
-
1–3 years of professional experience in quality assurance or software testing
-
Experience with automation testing tools such as Selenium, Playwright, or Cypress
-
Experience testing AI chatbots, AI Assistants, or agent AI systems
-
Solid grounding in QA fundamentals, including test case design and defect lifecycle management
-
Ability to test systems where outcomes are not always fixed or fully predictable
-
Strong written communication skills, particularly when explaining AI behavior to non-technical stakeholders
-
Ability to work independently and reliably in a remote setting
-
Familiarity with API testing tools such as Postman
-
Exposure to prompt evaluation or LLM output assessment methods