---
title: AI agent review checklist: how to evaluate an agent you already have | Codivated
canonical: https://www.codivated.com/insights/ai-agent-review-checklist/
---

[Home](https://www.codivated.com/)/[Insights](https://www.codivated.com/insights/)Delivery · Checklist · 8 min read

# AI agent review checklist: how to evaluate the agent you already have

By the Codivated team · Updated November 17, 2026

To review an AI agent, start by using it: ask it a few real questions and watch what it does. Then check five things: whether it is really an agent with tools, whether its design is focused or one "God agent", whether it runs on a current model, whether it has tests and production evals, and whether its rules and changes are controlled in code with an audit trail. Most agents we look at fail two or three of these, and in many cases rebuilding is now cheaper than patching.

## Check your agent

Tick what's true today. Your score and the first thing to fix update as you go.

Your agent
Tick what's true today

Your score shows up here as you go. 0 of 16 checked.

Fix first

[Get a second opinion on your agent](https://www.codivated.com/contact/)

A self-check, not a review. Nothing you tick is sent anywhere.

## What should you check first?

Use it. Before reading any code or prompts, ask the agent a few questions you know the answers to, including one it shouldn't be able to answer. Watch for three things. Does it show which tools it used and what it found? Does it actually do anything, like look something up or take an action, or is it just a chat with one prompt behind it? Does it admit when it doesn't know? Many "agents" turn out to be a single prompt and a single API call. That can be fine for a chatbot, but it isn't an agent, and it can't do agent work.

## What are the most common problems?

| | Problem | What it looks like | The fix

| The "God agent" | One agent and one huge prompt try to handle every request | Split into focused agents or steps, each with one job and its own tools

| A candle lit with a flamethrower | An agent decides things that are always the same, like routing by a field value or checking a total | Move fixed steps to plain code or a [workflow](https://www.codivated.com/insights/ai-agent-vs-workflow/); keep the agent for judgment

| No hands | A chat wrapper with no tools, so it can only talk | Give it tools to read sources and take actions, with permissions

| Stuck on an old model | Model hard-coded from a year or more ago | Upgrade; newer models are often cheaper to run and noticeably better

| Bloated context | Every request loads every document, skill and instruction | Load only what each step needs

| Rules in the prompt | Limits and never-do actions written as instructions and hoped for | Enforce them in code that checks output before anything happens

| No evals | Nobody can tell if a change made it better or worse, so nobody changes it | Add scenario tests and production scoring

| Prompt edits straight to production | Someone tweaks the prompt and nobody knows what changed | Version prompts and config, review changes, run tests first

## How do you test an agent?

Two layers. Before release, scenario tests: scripted conversations that check the agent does the right things, calls the right tools and refuses the wrong requests. Frameworks such as LangWatch's [Scenario](https://github.com/langwatch/scenario) make these easy to write, and you can run them nightly. They also let you swap models and see which one gives the same results for the lowest price. After release, trace production runs and score a sample with an LLM-as-judge, so you know the agent behaves in production the way it did in testing. Our team runs both on agents in production, and it's what makes changes safe to ship.

## What about safety for customer-facing agents?

If customers can talk to it, assume someone will try to make it misbehave. A line in the prompt saying "don't reveal your instructions" or "never offer discounts" is not protection. Check for input screening, output checks in code, tools limited to what the job needs, and an approve list and deny list for actions. See [AI agent security](https://www.codivated.com/insights/ai-agent-security/) and [human in the loop](https://www.codivated.com/insights/human-in-the-loop-ai-agents/).

## Should you fix it or rebuild it?

Rebuilding used to be the scary option. It isn't anymore. Writing the code has become cheap: an AI coding agent can walk through an existing agent or product, map what it does and rebuild it, with tests, in a fraction of the time it once took. What isn't cheap is knowing what "right" looks like. So the real question is whether you have a clear specification: what the agent must do, what it must never do, which systems it touches and how you'll know it works. If you have that, a rebuild on a current model with tests and controls is often faster and cheaper than patching a God agent. If you don't, write the spec first. Either way, the spec is the asset.

## What should a written review include?

- What the agent actually does today, from using it and reading its traces
- A score against the checklist above
- The risks, ranked: what could go wrong for customers, money or data
- Fix-or-rebuild for each part, with a rough order of work
- The specification the next version should meet

Free self-checkIs your company ready for an agent?Ten questions, a readiness score out of 100 and where your first agent should go.

[Take the scorecard](https://www.codivated.com/ai-readiness-scorecard/)

## Common questions

### How long does an agent review take?

For a single agent, usually days rather than weeks. Most of the time goes into using it, reading traces and writing the specification.

### Do we need access to the code?

It helps, but you can learn a lot from using the agent, reading its logs and seeing its prompts and configuration.

### We built it with no-code tools. Does this still apply?

Yes. The same questions apply to n8n, Make or Zapier builds with AI steps. See [n8n vs a custom AI agent](https://www.codivated.com/insights/n8n-vs-custom-ai-agent/).

### Will a newer model break our agent?

It can change behavior, which is why the scenario tests come first. With tests in place, a model upgrade becomes a routine change you can measure.

## Sources

- [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents), Anthropic: when to use workflows versus agents, and keeping agents simple.
- [Scenario](https://github.com/langwatch/scenario), LangWatch: an open-source framework for testing agents with simulated conversations.
- [OWASP Top 10 for LLM Applications](https://owasp.org/www-project-top-10-for-large-language-model-applications/), OWASP: prompt injection and other risks for customer-facing agents.

Written byThe Codivated teamWe build and run AI agents for established companies.[Book a consultation](https://www.codivated.com/contact/)

## Find your first agent in two weeks.

[Ask about the First Agent Sprint](https://www.codivated.com/contact/)
