AI QA Engineer

You own the gate into UAT. Builds that don't meet the standard don't reach clients. Our quality model has two layers. First, agents catch regressions at scale. Second, screenshot regression plus a human review catches visual and workflow drift on anything client-facing. You build both layers, run them, and answer for what gets through.

What You Own

• The UAT gate. You define, document, and enforce entry criteria for UAT on every release. You can block any build that doesn't meet the criteria. Overriding a block requires sign-off.
• Automated regression coverage. You build and maintain automated suites in CI so that regressions are caught before UAT, not by clients during it.
• Agent-based checks. You design and maintain AI agents that exercise workflows, generate test cases, and triage failures at scale. You also measure agent reliability by tracking false passes and false failures so the team can trust the results.
• Visual and workflow drift detection. You run screenshot regression across client-facing surfaces. Nothing client-facing goes to UAT without a human review pass.
• Quality reporting. You report on defect escape rate, gate pass rate, and flaky-test rate so engineering leadership can see release risk before it becomes client impact.

Who Relies On This Role

• AI engineers need fast, trustworthy feedback on their changes.
• Engineering Leadership
• Client Delivery and Customer Success needs clean UAT cycles that don't damage client trust.
• Enterprise clients need a platform that behaves consistently from one release to the next.

What Great Looks Like

• 6 Months. Gate criteria are documented and enforced on every release. Automated regression covers the top critical workflows in CI. Screenshot regression is live on all client-facing surfaces.
• 12 Months. Client-found defects in UAT are down from the current baseline. Agent checks run on every pull request.  Engineers treat the gate as a help rather than a bottleneck.

What You Do Not Own

• Fixing the defects you find. Engineers own fixes. You own detection, triage, and the gate.
• Client UAT execution and sign-off. Client Delivery and Customer Success owns this.
• The CI/CD platform itself. You build on it. DevSecOps and Infrastructure own it.
• Security and penetration testing. DevSecOps own this.
• Product requirements and acceptance criteria. Product owns these. You are expected to push back when something isn't testable.
• Production release go/no-go.

Required Qualifications

• 4+ Years in QA automation or software engineering in test, including owning automated suites.
• An AI-native working style: you use AI coding tools (e.g. Claude Code, Cursor) daily to build and maintain tests, and you can show it.
• Hands-on experience with modern end-to-end frameworks (Playwright, Cypress, or similar) and strong coding skills in Python and/or C#.
• Experience building or running LLM-based agents for testing, including a practical understanding of their failure modes, such as nondeterminism and false passes.
• Experience with visual regression tooling (e.g., Playwright snapshots, Percy, Applitools).
• The judgment and backbone to hold a quality gate under deadline pressure, and to explain why.

Required Qualifications

• Experience testing LLM-powered features, including building evals.
• Background in enterprise B2B or multi-tenant platforms with client-configurable workflows.
• Experience at a company moving from services to product.