Aziz Khan
← Writing

Why AI Agents Fail in Production and How to Fix Them

AI AgentsReliabilityDebugging

Agents rarely fail in the demo. They fail in week three, on an input nobody tried, at a time nobody is watching. And the failure is usually not the model being "dumb". It is something around the model that was never built.

These are the failure modes I see most when building agents and agentic workflows, with the symptom and the fix for each.

1. It says "done" and it isn't

Symptom: the agent reports success. The file was not changed, the email was not sent, the test never ran.

Cause: you are trusting the agent's summary of its own work.

Fix: verify with evidence the agent did not write. Check the database row, run the tests, diff the files. This is the reason Zippy review cards show the diff, test output and screenshots instead of a summary, and why a separate reviewer role checks the executor. I wrote more about it in why I built Zippy.

2. It loops

Symptom: a run takes ten minutes and a surprising amount of money. The trace shows the same tool called again and again with nearly the same arguments.

Cause: nothing in the loop says stop. The model keeps trying because trying is all it can do.

Fix: limits in code, not in the prompt.

MAX_STEPS=12
RUN_TIMEOUT_SECONDS=120
MAX_TOKENS_PER_RUN=200000

Also detect repeats: if the same tool call with the same arguments appears twice, stop and return an error. A failed run you can see is better than a quiet expensive one.

3. It picks the wrong tool or invents arguments

Symptom: it calls update_customer when it should search first, or passes an id that does not exist.

Cause: too many tools, overlapping tools, or descriptions written for the developer who built them.

Fix:

  • Cut the tool list. Merge tools that do nearly the same thing.
  • Rewrite descriptions to say when to use the tool and when not to.
  • Validate every argument on the server and return an error that names the next step: "No order with that id. Use search_orders first."
  • Take ids like tenant and user from the session, never from the model.

4. It worked on ten examples and fails on real data

Symptom: the first real customers send inputs that look nothing like your test prompts. Long ones, empty ones, another language, a blurry scan.

Cause: there is no eval set, so every prompt change is tested by trying two things by hand.

Fix: save real inputs as fixtures and rerun them on every change. In Phlo, scanned purchase bills are saved as test fixtures so the invoice reader is tested offline. Each production failure gets added to the set. After a few weeks the set describes your real traffic better than any prompt you could write.

5. The output breaks the code that reads it

Symptom: a JSON parse error at 2am, or a field that is sometimes a string and sometimes a list.

Cause: the code depends on free text that happens to look structured.

Fix: ask for structured output against a schema and validate it before use. In Uxlyze, Gemini fills a response schema and the prompts live in a versioned file, so a prompt change is a reviewed diff and not an edit in a dashboard. On a failed validation, retry once with the error, then fail the job cleanly.

6. It does something twice

Symptom: two refunds, two emails, two tickets.

Cause: a tool call timed out, the agent retried, and both calls went through.

Fix: every write takes an idempotency key, and the second call with the same key returns the first result. This is ordinary API design, and it matters more with agents because they retry on their own. In FoxlyFlow every send carries an idempotency key for the same reason. The background is in the sending limits post.

7. It falls over under load

Symptom: fine with one user. With fifty, you get rate limit errors from the model provider, timeouts and half-finished runs.

Cause: each request starts an agent run inline, and nothing controls how many run at once.

Fix: put runs on a queue with a rate limiter and a concurrency cap. The web request writes a job row and returns. A worker picks it up. Uxlyze works this way: the dashboard only writes a report row, a Go worker does the analysis, and a per-minute limiter means a burst waits in line. Work that cannot run yet should be queued, not dropped.

8. It follows instructions it found in the data

Symptom: the agent reads a web page, email or support ticket and then does something nobody asked for.

Cause: to a model, text in a tool result looks a lot like text from the user.

Fix: you cannot fully prompt your way out of this, so limit what a confused agent can do.

  • Least-privilege credentials. A read task gets read-only access.
  • Isolation. Zippy runs each ticket in its own git worktree so an agent cannot touch other work.
  • Human approval for anything irreversible or external. Zippy holds risky actions on an approval card.
  • Keep agents that read untrusted content away from tools that send data out.

How to debug the next one

All eight are hard to find without a trace. For every run, store the input, each model call, each tool call with arguments and results, and the final output. Then debugging becomes a routine:

  1. Open the failed run.
  2. Find the first step that went wrong, not the last.
  3. Decide which of the eight it is.
  4. Add the input to the eval set.
  5. Fix it in code where possible. Change the prompt last.

Step 5 is the one people skip. A sentence added to the prompt is a request. A limit, a schema or a permission is a guarantee.

If you have an agent that works in the demo and misbehaves with real users, this kind of audit and repair is part of my AI agent development work.