Production Agents Need Controls at Runtime
A Passing Test Is Only One Layer of Safety#
HiddenLayer examines two reported incidents involving agent-assisted work at Meta and Amazon. In the Meta case, guidance from an internal agent contributed to sensitive data being exposed to employees without appropriate authorization for about two hours. In the Amazon case, reporting linked an agentic coding tool to a long service outage after changes were made without the peer approval normally required for a human engineer.
The organizations disputed parts of the framing, and neither incident can be reduced to “the model failed.” That is precisely why the examples are useful. Permissions, approval rules, human actions, and runtime visibility all shaped the outcome.
The previous version of this article described a different incident and claimed the source documented failures after repeated scope expansions. It did not. The operational lesson supported by the source is that pre-deployment verification needs to sit inside a larger control system.
Repeat the tests when the system changes#
Pre-deployment tests still matter. Run them again when the model, prompt, tools, data sources, permissions, or workflow scope changes. Preserve the results so the team can show what was evaluated before each release.
But a passing suite only demonstrates behavior under the tested conditions. It cannot guarantee what an agent will encounter in production, particularly when the agent interprets goals and can choose among tools.
That limitation is not a reason to abandon testing. It is a reason to avoid treating a test result as the entire safety case.
Keep high-risk controls active during execution#
HiddenLayer recommends controls familiar from other production systems:
- least-privilege access for each task;
- explicit human authorization for irreversible or high-impact actions;
- distinct, purpose-scoped agent identities;
- staged rollout and sandboxing;
- records of tool calls, data access, and action sequences; and
- runtime enforcement that can block or halt unsafe actions.
These controls answer different questions. Testing asks whether known scenarios behave as expected. Least privilege limits the possible blast radius. Approval gates reserve consequential decisions for a human. Runtime evidence helps the team detect and reconstruct behavior. Enforcement allows it to intervene.
No single one substitutes for the others.
Test the control, not only the agent#
For a high-risk action, include the surrounding controls in the release test.
Try an action above the agent's permission level and confirm it is denied. Trigger a case that requires approval and confirm nothing executes before authorization. Revoke a tool permission and confirm the workflow fails safely. Verify that the agent's identity and action sequence appear in the audit record.
A control that exists in a diagram but has not been exercised is still an assumption.
Match the release scope to governance maturity#
When the team cannot yet observe sessions, enforce a stop rule, or separate agent permissions from human accounts, reduce what the agent can do. Keep it in a restricted environment, limit it to reversible actions, or use it to prepare work for human execution.
This is not a rejection of production agents. It is ordinary rollout discipline applied to systems that can interpret goals and act across tools.
The release decision should therefore use two forms of evidence: repeatable tests showing how the agent behaves in known cases, and active controls limiting what can happen when production presents something the tests did not cover.
Passing checks is useful evidence. Safe operation depends on what remains in force after the checks are over.
Acrein Group builds agent workflows with release evidence, scoped permissions, approval paths, and runtime controls as parts of the same operating system.