In the Arena #06

The Agent Is Not the System

Two architectures are emerging for agentic software. As the boundary between them moves, one property has to survive: reasoning and ultimate authority must remain separable.

Amanda Hua · September 23, 2026 · Operating knowledge from Apova

An agent may decide what should happen. The surrounding system still determines what can happen.

Everyone is building agents. Underneath the demos, two very different ideas about what an agent is are beginning to emerge.

In one, the agent becomes the unit of software: it reasons, acts, and — as the pattern matures — carries more and more of what decides whether its actions are permitted.

In the other, the agent is one intelligent participant in a larger system. It may decide what should happen, while authority, state, policy, evidence, and verification are enforced independently of it. The agent is intelligent, but the agent is not the system.

Both approaches make sense. They are not competing claims about what is correct; they optimize for different stages of the same transition.

The question worth asking is what happens as we move from today's software to a world in which most software itself becomes agentic.

What are the two agent architectures emerging?

The self-contained agent. Give the agent tools, memory, reasoning, capabilities, and its own operating instructions, and let it carry increasingly complete units of work. The attraction is obvious: composability, speed, portability, fewer handoffs, and a clean abstraction that scales by adding agents rather than by adding integration.

The serious version of this is not an agent with no external controls. Thoughtful proponents of highly autonomous agents still use external identity systems, policy engines, sandboxes, and audit tooling. The difference is one of degree, and it is the degree that matters: how much authority and operational responsibility is delegated into the agent and its runtime, rather than enforced independently of it.

The agent within a system. The model reasons and proposes. The surrounding system holds what must remain explicit: what the action is permitted to do, what state it operates against, which policy applies, what evidence supports it, what verification is required afterward, and what happens when it fails. Autonomy here is something granted and bounded rather than something the agent contains.

That is the whole distinction, and it is worth stating plainly rather than leaving it implied.

It is not about how capable the model is. It is about where authority lives.

What does each architecture trade away?

Self-contained agents deliver autonomy faster. They are easier to reason about as a unit, easier to move between environments, and easier to orchestrate as a population. The cost is concentration: the mechanism that decides what should happen is often the one that determines whether it is permitted to happen. That makes independent verification harder, widens the blast radius of a single reasoning failure, and complicates the question an enterprise is ultimately accountable for: under what authority did this action proceed?

Agents within systems cost more architecture. More integration surface, more explicit boundaries, more handoffs, less unconstrained autonomy. What that buys is containment: authority that can be narrowed or withdrawn without rebuilding the agent, state and evidence that exist independently of the agent's account of events, and recovery paths that do not depend on the failing component to diagnose itself.

Neither list is a verdict. Together they are the terms of a choice most enterprises are already making implicitly, often without naming it.

Why does the transition favor agents inside systems?

Because enterprises are not replacing everything at once.

They carry decades of databases, payment rails, permission models, APIs, approval structures, state machines, regulatory obligations, human review, and systems that cannot be rewritten on an AI roadmap. Agentic capability is entering that environment incrementally, one workflow at a time — which is the pattern I wrote about in #03.

In that environment, an agent that contains its own authority has to be trusted in one step. An agent that participates in a governed system can be trusted incrementally: narrow authority first, verified outcomes next, expanded delegation after. For many consequential workflows, that is not merely the preferable path. It may be the practical way autonomy gets introduced at all.

That is a claim about the present, not about the end state.

Does the boundary move as more software becomes agentic?

It probably does.

As larger portions of the stack become agentic, more of what currently sits outside the agent can plausibly migrate inward: richer context assembly, longer-horizon planning, coordination across agents, even some interpretation of policy. Agents talking to agents can absorb work that today requires an integration project. The self-contained approach becomes more viable as the surrounding environment becomes native to it.

So the line between the two architectures is not fixed. It is a function of how much of the environment is agentic, how much evidence exists about agent behavior, and how consequential the work is.

The boundary moves. It does not disappear.

What has to stay outside the agents?

However capable individual agents become, something still has to exist outside them — an independent observation and authority layer over a population of agents. Call it the supervisory plane: the part of the architecture that governs agents rather than participating as one.

It need not be a single centralized system, and its function is defined by independence rather than by size. Its essential property is that its authority and evidence do not depend on the agent being governed — which can be satisfied by several cooperating systems, by different authorities for different classes of action, or by supervisory components that are themselves agentic, as long as none of them derives its authority from the thing it is governing.

A supervisory plane should be able to:

• Observe the system from outside any single agent's own reasoning and representation of events.
• Constrain what each agent is permitted to do, distinctly from what it is capable of doing.
• Revoke or suspend delegated authority without waiting for the agent to cooperate.
• Correlate behavior across agents, so that systemic or coordinated failure is visible even when every individual agent looks locally reasonable.
• Preserve evidence and lineage independently, so the record does not depend on the agent that produced the outcome.
• Force checkpoints and intervene when local reasoning is insufficient for the consequence at hand.
• Stop execution — and have that stop take effect.

Each of those already has an analogue in enterprise infrastructure. Credentials get revoked. Circuit breakers trip. Transactions get reversed. Auditors read records the audited party does not control. Agentic systems do not remove the need for those functions; they raise the rate at which those functions may be needed and remove the human latency that has historically sat in front of them.

Is that just a kill switch?

I would avoid the phrase, not because the idea is wrong but because it makes people imagine one enormous red button — a single, blunt, all-or-nothing control that nobody wants to be responsible for pressing.

The useful version is graduated. Narrow an authority. Require a checkpoint. Suspend one capability while leaving the rest running. Quarantine a class of action pending evidence. Reverse a specific decision. Stopping everything is the last option in that sequence, not the design.

What actually matters is the property underneath it:

A system cannot rely exclusively on the entities being governed to govern themselves.

If the decision to stop an agent depends on asking that agent whether stopping is appropriate, the control is not independent. That holds when the agent is a language model with a database connection, and it holds when the agent is a sophisticated reasoning system supervising other agents. Independence is architectural. It is not a property that emerges from capability.

What does human oversight mean at that scale?

Imagine large populations of agents interacting with other agents — negotiating, purchasing, moving funds, modifying infrastructure, writing and deploying software, creating further agents.

At that scale, "human in the loop" cannot mean a person approving every action. That model does not scale, and pretending otherwise produces approval theater: queues cleared without review, which is worse than no review because it manufactures evidence of oversight that did not happen.

So humans move up a level. People define authority and the conditions under which it may widen. Systems enforce it at runtime. Supervisory functions monitor agent populations and handle most exceptions automatically. Humans intervene in the increasingly rare, increasingly consequential cases where judgment or accountability genuinely requires a person.

Humans do not leave the loop. They move above it.

That is a change in where human judgment is applied, not a reduction in human responsibility. Someone still has to own the delegation.

What happens when the reasoning layer itself keeps changing?

So far this has described boundaries moving over years. They also move at runtime, and that is the version enterprises are already living with.

A production system may route one task to a frontier model, another to a smaller open-weight model, another to something specialized for a narrow domain — and revise all of those choices as quality, latency, cost, privacy obligations, or regulation change. Teams operating at scale are optimizing across model quality, tool use, token efficiency, and cost at the same time, and the answers keep moving. In #05 I argued for choosing the capability each step actually requires. The further point is that the choice does not hold still.

So "the best model" is not a property of the system. It is a running decision.

That has a consequence for how the system is divided. If the reasoning layer is dynamic, the architecture around it cannot assume that today's split of responsibilities is permanent. Reasoning, authority, policy, state, evidence, tools, exception handling, and verification do not have to live inside one agent abstraction merely because an agent is technically capable of performing all of them. They should be separable enough that each can evolve on its own schedule.

Call it dynamic decomposition. Dividing a system into clear boundaries is ordinary engineering; what is different here is that the right division is itself a function of conditions that keep changing — the workflow, the consequence of failure, the models available, the economics, the latency budget, the regulatory context, the organization. The goal is not maximum separation, and not maximum agent autonomy. It is the right balance at a given moment, in an architecture that can move when the balance does.

Which is precisely why the invariants have to be stated separately from the topology. Models get swapped. Routing strategies change. Agents gain capabilities. Tools come and go. Through all of it:

Reasoning can interpret authority. Reasoning cannot self-authorize.

A different model should not acquire different execution rights because a router happened to select it. Policy, authority, state, evidence, and verification continue to determine what the system can do, regardless of which intelligence participated in deciding what it should do — the boundary I wrote about in #04, now applied to a reasoning layer that changes underneath it.

That separation is what lets a team optimize aggressively at the reasoning layer without destabilizing the execution layer. Seen this way, model routing is not only a cost decision. It is another demonstration of why production AI needs systems architecture around the agent.

The agent can evolve quickly. The system has to make that evolution safe.

Where does Agent Atlas sit in this?

We built Agent Atlas around the design principle underneath the second architecture: reasoning and execution authority are different concerns. Today, that means keeping context, policy, authority, state, evidence, exceptions, and lineage outside the model. As agent architectures evolve, the boundary can move without collapsing that separation. We designed it that way before broad autonomy was enabled rather than after, because separability is far easier to establish by design than to retrofit.

AI reasoning can be probabilistic. Enterprise execution must remain explicit, governed, and verifiable. In practice, those control surfaces are maintained by the execution architecture rather than inferred by whatever intelligence happens to be participating. The execution loop — understand, decide, authorize, execute, verify, resolve, learn — closes around the model rather than inside it, and what remains true around each step is the architecture.

A supervisory plane needs the same functions at a wider scope. Today, ours operate on bounded workflows rather than on a universe of agents: authority is granted per action and can be narrowed or withdrawn, state is explicit rather than inferred, evidence and lineage are preserved independently of the agent's own account, exceptions have defined paths, and verification is a required step rather than an assumption.

As agent populations grow, that separation extends outward — from governing execution inside one workflow toward observing and constraining many agents across many. We are building toward that deliberately, at the pace at which we can verify it. I would rather describe the direction honestly than claim the destination.

The architecture we need now is not the architecture of the end state

That is worth saying plainly. The design that makes autonomy adoptable inside today's enterprises — explicit boundaries, integration with existing controls, incremental delegation — is not necessarily the design that a predominantly agentic world will settle on. Architecture should be expected to move.

So it is worth being precise about where the commitment actually sits. Not in the topology. In the design principles the architecture has to preserve — its invariants:

• Reasoning and authority are different concerns.
• Consequential actions require explicit authorization.
• State, evidence, verification, and exceptions are real parts of the system, not assumptions about it.
• The governed entity should not be the sole source of its own control.
• Autonomy widens on evidence, not on assumption.

Those are design principles, not predictions about one correct future architecture. Where the boundaries sit, and how the system is composed, are engineering choices that should track conditions. Today, more of the control surface probably needs to live outside the agent, because enterprises are transitioning incrementally. As agentic infrastructure matures, some of that work — including some supervisory work itself — can plausibly move inward.

We are not opinionated about one fixed architecture. We are opinionated about the design principles that architecture must preserve. As agentic systems mature, those boundaries should move. The separation between reasoning and ultimate authority should not.

The right architecture is not static. The invariants are.

So the question that will matter is not how capable agents become. It is who governs them, with what independence, and on what evidence.

AI reasoning can be probabilistic. Authority over consequential execution cannot be. Authority has to come from somewhere other than the entity exercising it.

Even in a world where nearly everything is an agent, something still has to govern the universe they inhabit.

The agent is not the system.