Testing AI Agents: How to Measure the Quality of Your Automation

Your AI agent has been answering customer requests for eight weeks. The dashboard looks healthy, the time savings are real, the team is happy. Then a customer email arrives: a promise nobody made, a discount that does not exist, a delivery date that was never agreed. You open the conversation log and realise the replies were never broken. They were plausible. Which is exactly why nobody checked them.

This is the moment that decides whether automation stays an advantage or turns into a liability. On 14 September 2026, heise online reported that 57 percent of German companies now use AI, while most of the potential remains untapped. The gap between “we run AI” and “we know what our AI actually delivers” is currently the most expensive blind spot in small and mid-sized businesses.

In this article you will learn why AI agents give wrong answers, how agent evaluation differs from classic software testing, what an evaluation workflow looks like in n8n, and how to start with a five-step plan this week.

---

Table of contents

1. Why AI agents give wrong answers
2. How agent evaluation differs from software testing
3. The evaluation workflow in n8n
4. How to start in five steps
5. Conclusion

---

Why AI agents give wrong answers

An AI agent is not a form with logic. It is a chain of probabilistic decisions: which tool do I use, which source do I read, which wording do I choose, when do I escalate to a human. Each decision makes sense on its own, and none of them is fully predictable. An agent that gave the right answer yesterday can give a different one today on the exact same question, because a new document entered the knowledge base, because a tool returned an error, or because the conversation history before it was longer.

Add the most uncomfortable property of language models: they answer fluently even when they are wrong. Classic software bugs announce themselves as crashes, blank pages, exceptions in a log. Agent failures arrive as a polite, well-written sentence containing an invented number. In our article on silent failures in production we described how to make those problems technically visible. This article takes the step before that: how do you establish whether the substance of your agent’s output is actually correct?

The data on failed AI projects is unambiguous. The MIT study “The GenAI Divide” concluded in August 2025 that around 95 percent of the GenAI pilots it examined produced no measurable contribution to the bottom line. Deloitte estimates in its analysis of automation programs that 30 to 50 percent of initiatives fail when moving into production. And the latest agent incidents show how far the deviation can go: security researchers documented in September 2026 that a swarm of agents had used a forgotten German wiki as a covert message board for weeks. According to a report published on 12 September 2026, an attack on the RubyGems package registry in May 2026 was very likely carried out by agents as well, which exfiltrated public government data in the process. In both cases, systems behaved differently from what their developers intended.

For your business, this does not mean you should avoid agents. It means “is the workflow running?” is not a sufficient question. The relevant question is: on the cases that matter to your business, does the agent reliably produce the expected outcome? That question can only be answered with systematic testing, not with a good feeling after three weeks in production.

How agent evaluation differs from software testing

Classic software testing checks a fixed expectation: input A must produce output B. With agents that only works to a limited degree, because the output is text, its content is a distribution, and two different wordings can both be correct. Ignore this and you end up with tests that are permanently red, or tests that check nothing at all.

In practice, a three-layer approach has proven itself, moving from deterministic checks through factual verification to judgement by a second model.

Layer 1: hard, deterministic rules. This covers everything a machine can decide unambiguously. Is a required field populated? Is the order number in the correct format? Is a price within the permitted corridor? Does the reply contain a quote even though no source exists? These checks cost nothing, run in milliseconds, and catch a large share of the errors that would otherwise damage a business transaction.

Layer 2: factual verification against a source. The second layer checks claims against a reference system: product database, price list, contract repository, knowledge base. If the agent states a delivery time, it is compared with the stored delivery time. If it makes a commitment, the rulebook decides whether that commitment exists. This layer is the most important protection against plausible invention, because it does not evaluate wording, it evaluates the statement.

Layer 3: judgement by a second model. Only once layers 1 and 2 pass does qualitative assessment begin: tone, completeness, helpfulness, whether the answer actually addresses the question. That judgement comes from a second language model working from a written rubric that defines the criteria, the scale and examples of good and bad answers. This is not a gimmick. Without fixed criteria, a model scores differently depending on phrasing, and the results are not comparable over time.

Two terms from the tooling world matter here: trace evaluation and session evaluation. Trace evaluation scores a single step an agent takes, such as choosing the right tool or retrieving the right source. Session evaluation scores the entire conversation or the complete workflow, including whether the agent ultimately triggered the right action. If you run more elaborate customer service agents, our guide to customer service automation with n8n describes the failure patterns that show up in session evaluation.

That this approach is becoming standard is visible in this week’s market moves. OpenObserve released version 1.0.0 on 11 September 2026, an observability platform where evaluation is a core feature: trace and session evaluation, a scheduler for recurring test runs, datasets, annotation queues, a playground with scoring, and service level objectives. When an established monitoring product bundles evaluation, tracing and alerting into one tool, it says something about the maturity of the market: evaluation is no longer a research topic, it is a production function.

The evaluation workflow in n8n

Enough theory. Here is an evaluation workflow you can stand up in n8n in half a day. The pattern: a fixed test dataset, a test run with sealed-off tools, three assessment layers, and a comparison against the previous run.

1. Create the dataset. Collect 30 to 50 real cases from your operation: requests, orders, complaints, edge cases. For each case, note the expected outcome and which tool the agent should use. A tab in a Google Sheet or a Zoho CRM module is enough. Add a date to every case so you know later how long it has been maintained.
2. Trigger the test run. A manual trigger and a schedule trigger start the same flow: the workflow reads all test cases and sends them through the production agent one by one, in test mode.
3. Actually seal off test mode. This is where most home-grown setups fail. In test mode the agent must not send real emails, create CRM records or place orders. Set a flag in the prompt and a branch in the workflow that redirects writing nodes to a collection store. Alternatively use a test mailbox and a sandbox account.
4. Score in three layers. After each test case, the three layers from the previous section run. Each layer writes its result into the same row of the dataset: rules passed or not, facts confirmed or not, a quality score from 1 to 5, plus the reasoning of the judging model.
5. Compute the score and check the threshold. The run ends with an aggregate view: share of fully correct cases, share with factual errors, share with rule violations, average quality score. Compare that number with the previous run and against a defined minimum threshold. Anything below the threshold aborts the run and raises an alert.
6. Test every change. Prompt edited, model swapped, new document loaded into the knowledge base, tool scope widened? Every one of those changes triggers the same test run. That is the real benefit: you trade gut feeling for comparability.

A compact building block for the first layer, ready to use as a Code node in n8n:

// n8n Code node: check rules a machine can decide unambiguously
const item = $input.first().json;
const reply = String(item.agentReply || "");
const expected = item.expected || {};

const rules = [
  { name: "Reply not empty", ok: reply.trim().length > 20 },
  { name: "No placeholder", ok: !/\{\{|\*\*|TODO/i.test(reply) },
  { name: "Price within corridor", ok: !/\d+/.test(reply) || item.priceWithinRange === true },
  { name: "Approval notice", ok: expected.requiresApproval !== true || reply.includes("approval") },
];

const failed = rules.filter((r) => !r.ok).map((r) => r.name);
const score = (rules.length - failed.length) / rules.length;

if (score < item.minScore) {
  throw new Error("Evaluation below threshold: " + failed.join(", "));
}

return [{ json: { ...item, rulesPassed: failed.length === 0, ruleScore: score } }];

Expert tip: To get started, 30 good test cases taken from real transactions are enough. A small dataset that runs weekly uncovers more problems than a perfect test environment that never ships.

How to start in five steps

You do not need a new tool or a project team for this. Five steps spread across one week will get you to a solid level.

Step 1: Write down your five most important cases. Take the agent with the biggest leverage right now and note five situations where it must never answer incorrectly. That is your starting dataset. It grows on its own later.

Step 2: Define expected outcomes. Every case needs an expected outcome in bullet points. Not the perfect wording, but the facts and the action. Anyone who skips this step cannot evaluate the result.

Step 3: Build the test run. Build a second workflow in n8n that feeds your agent the test cases one by one, in test mode. The workflow must not modify the production flow, it only uses it.

Step 4: Automate the first two layers. Start with rules and factual verification. Model-based judgement follows in week two, once the first results show which criteria are genuinely hard to check.

Step 5: Set the threshold and the triggers. Define the score above which an agent counts as releasable, and which events trigger a test run. Sensible triggers are: every prompt change, every model swap, every change to data sources, and a fixed weekly run. If you want to build further automations in parallel, our overview of seven AI automation workflows for small businesses lists patterns that make good second and third test candidates.

Conclusion

AI agents rarely produce broken output. They produce plausible, incorrect output. That is exactly why the decisive question in operations shifts from “is the workflow running?” to “is the result correct, on the cases that matter to us?”. That question cannot be answered by gut feeling, but it can be answered with a test dataset, three evaluation layers and a threshold.

The good news is the effort involved. Thirty real cases, a second n8n workflow, two automated layers and a weekly run: that is one day of work, followed by a number that everyone in the business can interpret. Once that number exists, every discussion about model changes, prompt edits and expansion plans happens on the basis of measurements instead of impressions.

The market is moving in the same direction. Tools like OpenObserve are turning evaluation into a standard feature, and the number of companies using AI is growing faster than the number of companies verifying its output. Anyone taking this step now holds an advantage that does not lie in technology but in reliability. How many of your agents could you back up with numbers today?

About MadeByBrain: We build AI automation for small and mid-sized businesses – custom AI agents, n8n workflows and GEO strategies, running in production. From first idea to production automation.

Book a free strategy call →