← Back Operate · Acrein Group

Your Agent Is Still Doing the Old Thing

5 min read · Acrein Group

When Your Agent Keeps Following Instructions That Changed

Your agent just pulled a document from your knowledge base. The document said to do one thing. Your team changed that process two months ago, but the agent did not get the memo.

It is not broken. It is working exactly as designed. That is the problem.

The agent retrieved a source. The agent trusted it completely. The agent acted on it with full confidence. And now something in production is wrong in a way that does not fit any incident category your team has ever used.

Most production incidents in agentic systems look clean from the outside. The system behaved as built. The workflow executed as designed. No alerts fired. The agent logged every step. But somewhere between what the agent retrieved and what it did, reality shifted and nobody noticed.

That shift lives in the knowledge the agent acted on.

The agent was right. The information was wrong.

You updated the process. Marketing changed the approval flow. Finance changed the escalation thresholds. A junior team member fixed a bug in the customer data three weeks ago and the fix conflicted with a doc nobody had touched in six months.

All of this is true and the agent knows none of it.

It retrieves the document that was in your system when you deployed it. That document is still in the index. Nothing told the agent it was stale. Nothing told it to check. It reads the instructions. It acts on them. It moves to the next task.

The failure is not in the agent. The failure is in the knowledge.

You can feel this in the operations now. The agent behaves correctly but wrongly. It does things that make sense if you read the document it was following and no sense if you know what your actual process is now. Your team can trace back and see it. The agent has no way to see it like it would in a traditional incident review.

Knowledge does not stay true on its own.

Documentation drifts. It happens everywhere, but it compounds when machines are reading it.

A human sees a doc that does not match reality and either updates it or makes a mental note. An agent has no mental notes. It has retrieval. It fetches the source it is allowed to reach and it trusts it the way humans trust the floor beneath them.

When that source stops being true, the agent does not slow down. It does not second-guess. It does not ask another system or bump it to a human for review. It acts on stale information at machine speed with complete confidence.

You have sources in your knowledge base right now that your team has not looked at since before this agent went live. You do not know which ones. Neither does the agent.

Processes changed. Thresholds shifted. A form field was deprecated and nobody deleted the doc that still referenced it. One team updated their approval flow and another team's runbook still described the old one. A source was right yesterday and wrong today and will be right again next week when someone remembers to sync it.

The decay is invisible until a confident wrong action surfaces in production and your team spends hours tracing back to find out that the agent was just following instructions that stopped being true.

Nobody owns the knowledge the agent runs on.

You have ownership everywhere else. Someone owns the agent. Someone owns the workflow. Someone owns the tool. Someone owns the cost and the decisions it makes.

Almost nobody owns the sources the agent retrieves from.

Those sources are dependencies. They are as much a part of your agentic operation as the model running it or the workflow calling it. They are live dependencies that change. They drift. They contradict each other. They become obsolete without warning.

But they have no owner. No review cycle. No cadence. No one person responsible for keeping them honest.

You treat your knowledge base like a one-time input. You build it. You load it. You deploy the agent. The knowledge is done. It is not done. It is only done if someone owns it and maintains it the way you maintain any other live system.

That person does not exist in most organizations. That gap is where the failure compounds.

What a knowledge incident looks like.

A knowledge incident is not a system failure. It is not an alert. It is not a stack trace. It is the moment when an agent acted on information that was not true and that action had a real consequence.

When that moment arrives, your team should be able to answer these questions immediately.

What did the agent retrieve? Not what it did. What source did it actually read?

From what document or system? When in the workflow did it pull that source?

When was that source last verified as correct? Not when was it written. When was it last checked against reality?

Who owns keeping that source honest? Not who wrote it. Who is responsible for maintaining it right now?

These are not audit questions. They are operational questions. They are the difference between a knowledge incident you can see clearly and a knowledge incident that looks like the agent is just broken.

Start this week by picking one source your agent retrieves from most often. The one it hits for every workflow. Ask your team those four questions about it. If you cannot answer all four, you do not own that knowledge yet.

The work is boring and it matters.

The operational lesson from running agents at the workflow level is not dramatic. It is not about architecture or latency or model choice. It is this: the hard work is not building an agent that retrieves information and acts on it.

The hard work is keeping the information honest.

You cannot fix this with a better retrieval system. You cannot fix it with guardrails that tell the agent to double-check. You cannot fix it by adding more verification steps to the workflow. The agent still retrieves the same stale source. The only question is whether it acts on it or wastes time verifying it is stale.

You fix it by assigning ownership. By defining a review cadence for every source the agent can reach. By treating knowledge decay as an operational problem you prevent, not an incident you debug after it breaks.

That is boring work. It is also the difference between an agentic operation that scales and one that amplifies your mistakes at machine speed.


Start by owning the knowledge your agent acts on. Acrein Group builds and runs this ownership structure into the operations it scales. The teams that move fast with agents are the ones that solved this first: the agent is only as honest as the knowledge it retrieves.

Building, stuck, or ready to scale?

The right conversation at the right moment changes everything. Let's have it.

Talk to us