AI Agentic Tester
Delan Associates, Inc
- Location
- Remote (Denver, MO)
- Employment
- Contract
- Level
- Senior Level
About the Role
Delan Associates is seeking an AI Agentic Tester to validate the accuracy, consistency, and functionality of AI agent workflows and LLM responses. The role focuses on ensuring high-quality AI solutions through rigorous testing of RAG pipelines, multi-agent interactions, and end-to-end applications.
Skills
Full job details
AI Agentic Tester
Denver, MO (Remote)
Must-Have Skills
* AI Agentic Testing
* Functional Testing
* Generative AI Testing
* Large Language Models (LLMs)
* AI Agents & Agentic AI
* Retrieval-Augmented Generation (RAG)
* Prompt Engineering Validation
* AI Model Validation & Evaluation
* API Testing (Postman, REST APIs, Swagger)
* Python
* Test Automation (Selenium / Playwright / PyTest)
* AI Evaluation Metrics (Accuracy, Hallucination, Relevance, Consistency)
* End-to-End Workflow Testing
* Multi-Agent Testing
* Azure OpenAI / AWS Bedrock / Gemini
Core Responsibilities
* Perform functional testing of AI agent workflows and end-to-end AI applications.
* Validate LLM responses for accuracy, relevance, consistency, and hallucination rates.
* Test RAG pipelines, prompt engineering, and AI agent orchestration.
* Execute API testing using Postman, Swagger, and REST APIs.
* Develop and maintain automation scripts using Python, Selenium, Playwright, or PyTest.
* Validate multi-agent interactions, tool calling, and workflow execution.
* Apply AI testing methodologies and evaluation metrics to ensure response quality.
* Work with Azure OpenAI, AWS Bedrock, Gemini, or similar AI platforms to validate AI solutions.