Guard rails that survive agent speed
How teams bounded agent identity and access inside delivery, and why rebuild cadence keeps losing to feature work.
In this recap
The room
Twelve senior practitioners, one morning, one table, and a bigger turnout than the format usually takes. Platform and CI/CD leads, principal engineers, engineering managers, an SVP of engineering, an IT director and a managing director, drawn from investment banking, wealth management, a building society, e-money, shipping and logistics, grocery retail, construction, web infrastructure, edtech, pension insurance and two software firms serving regulated markets.
The session ran under Chatham House convention. Everything below is reported without names or organisations, which is the condition that lets people describe outages and near misses honestly rather than in the version that would survive a press office.
This was the fourth room in the supply chain series, and the last one to run AI alongside it. The planned separate AI forum had been folded into this session rather than run on its own over the summer, and by the closing round the room had asked several times over to pull the two apart. The AI Governance track that started in August came directly out of this morning.
The rebuild treadmill
The thread that opened the session and kept resurfacing: software is perishable, and most estates do not treat it that way.
One participant described a rule requiring everything to expire and be rebuilt every ninety days, set against software his firm had not released in something like four and a half years. Rebuilding is not the hard part. The hard part is that a rebuild of software nobody has touched in years is a delivery project wearing a maintenance badge, and it competes for the same people as the roadmap.
The second half of the same problem is what the scanners now surface. AI-driven scanning finds more, which turns into security work that displaces planned delivery rather than sitting alongside it. Several people recognised the pattern: the backlog is not the vulnerabilities, it is the remediation capacity behind them.
Treat software as perishable, and rebuild on a fixed cycle whether or not anything has changed.
A position argued in the room, reported without attribution.
When the agent went further than anyone meant
This produced the sharpest stories of the morning, and they were told plainly rather than defensively.
One described an agent shutting down production services in the belief that they were pre-production. Another described being told to point a coding agent at a list of security findings, and watching it re-enable code that had been deliberately commented out. Neither was a failure of the model. Both were failures of scope.
What followed was the most practical stretch of the session, because several people had already built something.
One team described an application catalogue sitting on top of their internal platform, tagging each application as business critical or experimental. A business-critical production application cannot be deleted regardless of the permissions the caller holds. A human has to move it to decommissioned first. The framing that landed with the room was that this is permission-based control rather than tag-based: the tag is not advisory metadata, it changes what the platform will allow.
Another participant described the same approach running inside their own organisation. Agents cannot commit on their own. Agent identity is deliberately separated from the calling user, so an agent does not inherit whatever the person who invoked it happens to be able to reach, with shadow identities and internal scoping modules limiting the blast radius. Sandbox use has been extended to non-technical staff, on the reasoning that the people least likely to recognise a risk are the ones who need the guard rails most.
Should AI-generated code be treated as output from a trusted contributor, or as something arriving from an unknown source until proven otherwise?
One view held that generated code should be treated as a threat actor rather than a developer, because the training data and provenance are unknown and the copyright position is unresolved. The counter-view was that the tool is only as good as the operator, and that a skilled engineer who knows what they are asking for produces reviewable work either way. The room did not resolve it.
AI in the SDLC, or an SDLC for AI
The room kept pulling apart two problems that get discussed as one: using AI inside delivery, and governing AI itself as software that still needs the old guard rails.
On the second, the honest reading offered was that the guard rails already exist and are simply not being enforced with the same rigour, because the rush to adopt outran the process. Nobody argued that a fundamentally new discipline was needed. Several argued the existing one is being waived.
A second governance problem came up that nobody in the room had solved: software-as-a-service suppliers shipping AI features switched on by default. The work is not evaluating a new product, it is working out whether you even want the feature that has arrived inside a product you already run. That thread carried into the AI Governance session a month later, where it drew the strongest recognition in the room.
Proving any of it works
One of the clearest problems stated all morning, and one with no answer offered.
An organisation running enterprise coding assistants across its engineers reported high adoption and heavy token consumption, and no demonstrable productivity gain behind it. Senior engineers do well. Others produce, in the phrase used, a lot of slop. The blocker is not the tooling and not the licence cost. It is that no pre-AI baseline was captured, so there is nothing to measure the present against.
Several people recognised this immediately. Once adoption is already widespread, the counterfactual is gone.
Volume, speed and time to exploit
The case made to the room was that the arithmetic has moved, and nobody pushed back on the direction.
The numbers put forward: on the order of two hundred CVEs a day, a national vulnerability database that has effectively stepped back from the work, and European vulnerability and exploitable-vulnerability databases arriving with a deadline attached. On average around sixteen percent of code in a given estate comes from independent open source, rising towards the whole codebase in some languages, which is the argument against dependency-only scanning and for scanning the full binary.
The sharper point was speed rather than volume. Frontier models raise both the rate at which vulnerabilities are found and the rate at which they are exploited, with time to exploit approaching zero. A vulnerability can be exploited before it is disclosed, which breaks the assumption underneath every patch window.
What was proposed against it: curation rather than open consumption, trust metrics and provenance on third-party dependencies, and upgrade and incident-response policy codified so it can actually be automated. One suggestion went further, holding releases back until validated.
Trust, determinism and who carries the liability
The most enjoyable thread of the morning, conducted almost entirely in analogies, and underneath them a serious question: how much determinism does a business need before it can rely on what comes out?
The power drill did the most work. A drill lets a skilled person do more, faster, but it does not tell them where the screw goes. On that reading AI expands what an expert can do and does nothing for someone who is not one. Set against it was the plumber you have never met and the flat-pack wardrobe you assembled yourself: at some point you rely on work you cannot personally verify, and the question is what evidence makes that acceptable.
Companies are being built on faith rather than fact.
A position argued in the room, reported without attribution.
Data sovereignty sat underneath this. One participant described the practical version of the problem: whether to send customer data to AI suppliers at all, scepticism that they are as far along as they claim, and the need for an industry-consensus position to take to a board rather than a personal judgement call. She also noted an asymmetry worth sitting with. Clients will accept a large incumbent handling customer data largely because of who it is, while a start-up supplier is asked for heavy legal assurances that the data will not be trained on.
When the toolchain is the single point of failure
Raised by someone whose job is resilience, and it visibly landed.
The evidence offered was a hosted platform showing degradation on eighty-nine of the previous ninety days. The exposure is not the degradation itself, it is what happens when an emergency change needs hosted runners to build and they are not there. Keeping on-premise runners in sync as a standby is its own maintenance burden, and one that quietly rots when it is never used.
The comparison drawn was an airline incident where a fourth-party failure took check-in down and staff fell back to pen and paper. It worked because a manual fallback existed. A regulated financial platform has no pen-and-paper mode.
What is your backup plan when the core toolchain is unavailable and you need to ship an emergency change?
It was put to the room directly and no answer is recorded against it. That is worth noting rather than smoothing over: a room of twelve senior engineering leaders, and the record carries no described fallback.
Where the room did not agree
Four threads closed the morning unresolved. They are the most useful thing in the record, because each one is a decision a team somewhere is making this quarter without a settled answer to point at.
Whether AI-generated code should be treated as arriving from an unknown source, or as ordinary output from a tool whose operator remains accountable.
Whether to adopt now on customer data or wait for the market to settle. One reading was that this is early cloud repeating itself, with the same sovereignty concerns raised then and largely faded since, and that self-hosted or local frontier models resolve most of it. The other was that the suppliers are not as far along as they claim and a board needs more than an analogy.
Whether AI needs its own delivery lifecycle, or whether the existing guard rails are adequate and simply are not being enforced while everyone rushes to adopt.
Whether the cost of all this resolves itself as the market grows and open-weight models get cheaper, or whether it is a structural constraint that has to be designed around now.