Autonomy Scales Exposure Before It Scales Return

We are treating agents like the old rules no longer apply. They do.

There is a familiar smell in the air again for those of us who come from cyber security. Not only because new attack surfaces are emerging, but because the old seduction has returned wrapped in the language of intelligence, progress, and inevitability.

Anyone who has spent long enough around security knows the scent immediately. Excitement. Convenience. Overconfidence. Selective amnesia. The old seduction in new clothes: this technology is too important, too strategic, too extraordinary to be slowed down by the disciplines that governed everything that came before it. We will secure it later. Govern it later. Tighten the permissions later. Clean up the observability later. Sort out the audit trail, the revocation paths, the blast radius later.

Later is always where the damage begins.

What unsettles me about the current agent conversation is not just the hype. It is the strange implication, usually left unstated, that autonomous systems somehow sit above the hard-won rules of networked systems. It is the enchantment of agency: the visible leverage, the fact that a system appears to act, coordinate, deliver, and create value in ways that feel qualitatively different from traditional software, and the temptation to treat it as exempt. As if a model calling tools, retrieving files, passing work across systems, carrying memory, crossing organisational boundaries, and acting on behalf of users does not also create new paths for compromise, outside influence, and control to enter the firm.

It does.

And in many ways, it is the old problem with more reach, more speed, and more authority.

I say that as someone shaped by years in cyber security, watching firms bleed through exposed storage, weak identity, inherited permissions, over-trusted service accounts, over-broad credentials, soft internal trust, and architectures that treated “inside the perimeter” as a synonym for “safe.”

The difference is that the systems being rushed forward today do not merely store information or display it. They interpret it. Route it. Summarise it. Act on it. Carry it across boundaries. They retrieve, synthesise, call tools, trigger follow-on steps, and reshape the conditions under which people decide what to trust.

That should make the room colder than it currently is. Yet even after a major agent deployment was publicly breached through a weakness as old as the internet itself, the reflex is the same: that was their architecture, their controls, their sloppiness. Ours is different.

Because the fantasy right now is that multi-agent systems are mainly a story about seamless coordination, reduced friction, and faster execution. The demos encourage that fantasy because they are designed to hide the mess. Clean handoffs. Clear outcomes. No politics. No inherited permissions chaos. No ancient service account still alive from a project nobody remembers. No brittle dependency buried three tools deep. No legacy integration everybody relies on and nobody wants to touch.

Then the agents land inside the real operating environment, and all the buried conditions start to matter.

Shared drives nobody meant to expose.
Internal tools sitting on broad standing trust.
Service accounts nobody can fully map.
Tokens with more scope than anyone intended.
Prompt layers treated like setup rather than live operating infrastructure.
Thin logs. Fuzzy attribution. No clean reconstruction of what actually happened.
No clean answer to who can pause what, stop what, or revoke what when it matters.

Worse, the pilot graveyard matters too: forgotten prototypes, still live, still wired into enterprise data, still exposed long after the organisation forgot they existed.

Because autonomy does not arrive on clean ground. It lands inside whatever the organisation already is: its permissions sprawl, its vague ownership, its accumulated workarounds, its uneven internal boundaries, its unexamined trust assumptions, its escalation dynamics, its habit of calling something temporary until it becomes structural.

Once agents start moving across that terrain, ordinary weaknesses stop being local defects. They become shared exposure.

The upside is real. That is exactly why the conditions for trust, control, and intervention have to be designed before the architecture earns the right to scale.

That is why I do not trust the breezy confidence around multi-agent ROI. Not because I think autonomy is fake, or because I doubt the upside. I have helped realise it.

Because exposure arrives first.

Before the return is legible, the blast radius is already widening. Before the operating gain is measured, the trust surface is already expanding. Before the board gets its efficiency story, the firm may already have created a new locus where access, judgement, files, prompts, workflows, and action accumulate faster than control does. That is what an internal AI platform becomes: not just a productivity tool, but privileged infrastructure legitimised in the language of intelligence, progress, and inevitability.

The deeper problem is that firms will not actually want narrow agents for very long. They will want agents with power and reach, and power and reach pull in a direction: toward breadth. More context. More systems. More files. More delegated authority. More room to traverse the firm like a highly privileged insider.

That is where the posture starts to rot. Not because anyone writes “god mode” into the architecture diagram, but because the demand for agentic power and reach keeps pulling the system in that direction.

And that pull is not only technical. It is organisational.

Once the agent starts to resemble a highly privileged insider, the central question changes.

Not what can it do?
Who does it serve?
Whose instruction counts?
What authority is real?
Who can overrule it?
Who can stop it before “helpful” becomes “material”?

That is where the current conversation falls short. Agents do not only widen the technical attack surface. They force organisational ambiguity into the open. Questions of authority, legitimacy, and intervention that were previously blurred or deferred now sit much closer to action. If those questions are not settled around the system, they will not be settled by the system.

We know this pattern already. Give any system enough convenience, enough implicit trust, enough inherited access, and sooner or later someone starts arguing that tighter controls are getting in the way of speed and value. That is how over-permission gets rationalised. That is how exceptions become permanent. That is how blast radius expands while everyone is still talking about productivity. We learned that the hard way over decades, and some of us spent those years cleaning up after exactly this kind of amnesia.

Now we are acting like agents deserve a special exemption from that memory.

They do not.

If anything, the standard should be harsher.

Because the system is no longer just sitting inside a bounded environment waiting to be queried. It may be retrieving from internal and external sources, carrying memory, calling tools, synthesising responses people trust, handing work to other components, reaching out to other models, drawing on external research, and moving with low-friction access across boundaries that used to require more explicit human intention. Once those boundaries are being crossed routinely, the question is no longer only what the system can reach out to, but what can reach back.

An agent is not a special exemption from the rules. It is a force multiplier on whatever was already weakly governed. And the more power and reach you want it to carry, the harder it becomes to avoid the oldest security sin of all: broad access justified in the name of speed and value.

Every inherited service account.
Every over-scoped token.
Every half-understood internal trust relationship.
Every editable prompt layer.
Every weakly governed tool connection.

All of it becomes more dangerous the moment an agent can traverse it.

And in multi-agent settings the problem gets worse again.

Because weakness starts to travel.

A bad instruction does not stay put.
A spoofed identity does not stay local.
A compromised prompt does not remain one component’s problem.
A confused authority signal does not die in one thread.

It propagates. It gets repeated. It gets reinforced.

One agent’s output becomes another agent’s premise. One local failure becomes a cascading failure with worse attribution and wider consequence.

Responsibility diffuses faster than return compounds.

That is the part much of the current market language leaves firms poorly prepared to face. They still talk about agentic architectures as if coordination itself were proof of maturity. It is not. A chain of agents deftly handing work to one another is not evidence of control, no matter how impressive. It may simply be a more decorative way of mistaking motion for maturity.

Least privilege still matters.
Identity still matters.
Per-request authorisation still matters.
Auditability still matters.
Revocation still matters.
Segmentation still matters.
Policy integrity still matters.

Prompt integrity matters far more than most firms are prepared to admit. Because once the prompt layer shapes retrieval, interpretation, delegation, or action, it stops being a quirky implementation detail. It becomes part of the operating environment. And if it can be altered without control, the question is not simply whether the system still works. The question is whether the agent you trusted is now carrying someone else’s intent through your systems with your permissions.

That is not theoretical. A very public example surfaced recently. The outcome was contained not by design, but because the people who found it chose disclosure over exploitation.

That is what should keep serious people awake.

And this is where the ROI conversation becomes adult again.

The reason measurable returns appear first in boring, bounded deployments is not only that they are easier to measure. It is that they are easier to trust. Easier to supervise. Easier to contain. Easier to reverse. Easier to understand when something goes wrong. Easier to shut down before the damage spreads.

That is what maturity looks like.

Not the number of agents in the architecture diagram.
Not how smoothly they pass work between them.
Not whether the demo looked like the future.

Maturity is when a firm can answer, without flinching: what does this system have access to, who approved that, whose interest is it allowed to serve, what can it change, how do we know what it touched, and who can kill it before “concerning” turns into “material.”

If that sounds harsh, good.

Because some of the current language around agents has the emotional texture of the early internet: expansion first, control later, with a faint undertone of destiny that makes basic discipline sound small-minded.

I’m old enough to remember that atmosphere. Many of my peers do too.

I do not want to watch this industry walk back into that wild west era with better demos and worse memory.

The difference now is that these systems are closer to decisions, closer to internal knowledge, closer to customer interactions, closer to capital, and closer to action. The stakes are higher this time. Which means the lesson should not need to be relearned. But it probably will be. Markets love amnesia when the promise is intoxicating.

Years in cyber security leave you with a particular instinct. You stop being impressed by motion alone. You become slower to trust the demo, slower to trust the language, slower to trust the confident abstraction that arrives before the operating conditions are clear. You learn to stay close to the real environment in which the system must live: the workflow, the permissions, the ownership, the costs, the failure modes, the constraints.

That instinct has shaped all of my work in AI. Not because I am anti-AI. Because I am serious about it. I believe in the technology, the upside, and the value it can create. I have helped make that value real for organisations. But I have also spent long enough around security to know what happens when excitement outruns control, when power outruns ownership, and when systems are trusted before the conditions for trust have actually been designed.

The further the conversation drifts into hype, demos, abstractions, and promises detached from operating reality, the easier it becomes to make expensive mistakes with great confidence. The closer you stay to the real conditions of deployment, the better your judgement gets.

That is why I keep coming back to the same hard truth. The fastest path to durable ROI is not autonomy first and control later. It is authority, identity, containment, and intervention designed in from the start.

Without that truth, you are not building leverage.

You are building a system waiting to be owned.

--

Stuart Winter-Tear

Independent advisor | Author of UNHYPED

Advisory: Executive Calibration