Last week I participated in Thoughtworks’ Future of Software Development retreat. This was an invited gathering run under the Chatham House Rule, which is why I can tell you what was said as long as I don’t tell you who said it. Over a day and a half of Open Space sessions (and adjacent socializing), rooms full of people who build and advise on software argued about a lot of things: whether a “dark factory” that ships code with no human in the loop is real yet, whether code design still matters when maintenance is the agents’ problem, how agents should coordinate with each other, how to modernize a COBOL system nobody fully understands anymore. The sessions had different titles and different casts, and on the surface they were about different problems.
But they weren’t. Nearly every one of them was a different facet of the same argument. How much do we let an agent decide, and how do we stay confident in what it does?
I saw it wherever I looked. One session laid out an explicit spectrum from fenced-off industrial robots to a fully autonomous “dark factory,” and asked where on that spectrum our own systems should sit. Which is just the delegation question drawn as a diagram. Another was preoccupied with the human review-and-acceptance step between an agent producing code and that code shipping, and how it’s becoming the place work piles up now that generation is effectively free. The delegation question as a bottleneck. A third took its title from the old management “bring me a rock” parable, about a boss who can’t say what they want but insists they’ll know it when they see it. The room quietly turned it from “how do we explore by elimination when an iteration costs minutes” into “who should be allowed to steer the model at all,” which is the delegation question as a permission problem. (Sam Ruby, one of the participants, describes that same turn in his own writeup, and it’s worth reading.)
I want to spend this post on the argument underneath the arguments, because I think many of the disagreements we wrangled over were mostly not disagreements about principle. They were about confidence, and we were not being at all precise about what that word means. I’ll try to be, because getting precise about it is what set me thinking about how to reason about these workflows systematically.
An uncomfortable consensus
Let me start with something that we actually seemed to have a strong consensus about, more than I see in wider industry discussions outside this event.
Many, many people I hear from in the industry strongly believe that line-by-line human code reviews, embodied in the pull request, is the indispensable wall standing between us and an agent doing something stupid at scale. Keep a human reviewing every diff, the thinking goes, and you’ve kept control.
The room did not believe this. Not because these people don’t care about quality - several of the most skeptical voices were the ones who’d reviewed the most code in their careers - but because, pressed on it, nobody could defend the premise. One session ended on the blunt observation that no one could cite hard data on how many real defects manual review has ever caught, and that the practice survives on an illusion of safety more than on measured evidence. A separate session, coming at it from the opposite direction, came to the same admission: that human review had probably never been as effective as the industry liked to believe, and that the honest question isn’t how to preserve it in the agentic era but how to build something that earns trust in its place.
Sam Ruby, writing afterward, put his finger on why it matters: code review and security audit were supposed to be the last duties you’d hand to a machine, the judgment of last resort. And instead, in room after room, they turned out to be among the first to go.
Here’s the part I want you to hold on to, though, because it’s easy to hear “review was never the guardrail” as “stop reviewing,” and that’s not it. The problem was never whether to review. It was what we’re reviewing, and how closely. Reading every line was already infeasible at the volume agents produce. But more to the point, several people described their real question shifting out from under them: not “is every line here correct” but “does this deliver the outcome the user actually needed.” The scrutiny didn’t disappear. It moved up to a higher level.
Which raises the obvious question: if not the line of code, then what are we actually checking?
Where the rigour goes
Here’s an answer that came up in one form or another across most of the sessions: the rigour doesn’t go away when an agent writes the code. It moves.
Sam Ruby put it well: the discipline we used to apply by reading code migrates to the ends of the process instead. It goes upstream, into how carefully we frame the thing we’re asking for. And it goes downstream, into the checks that decide whether we got what we wanted. Neither of those is line-by-line review, and both matter more if you’re stepping away from reading the code itself.
Take the upstream end first. One participant in the “bring me a rock” session said they’d happily delegate almost anything to an agent, including the conceptual model of the system, but that they drew a hard line at the acceptance criteria: the description of what a correct result actually looks like. That’s the one thing, they argued, an agent can’t work out on your behalf, because it’s the statement of intent everything else is judged against. Several people arrived at versions of this from different directions. The work you do before you hand anything over, getting the objective clear and stated in a way that can be checked, is doing more and more of the job.
The modernization crowd had the sharpest example of this. Across several efforts to translate ancient COBOL into something modern, the common thread wasn’t the translation at all. It was building a conformance suite first: a body of tests derived from what the legacy system actually does in production, that both the old and the new system get run against. When the conformance tests and the written specification disagree, the tests win, because they capture real observed behaviour and the spec only captures what someone remembered to write down. Several people described letting multiple agents work in parallel on different parts of a rewrite, and the thing that stopped those agents drifting into mutually incompatible designs was that shared, executable definition of “still behaves the same.” The preparation was the control. More control made it safer to increase autonomy.
Now the downstream end. This is where a lot of the practical energy in the event actually went, under a word that’s rapidly wearing out from overuse: the harness. Everything you put around an agent to shape how it works and to check what it produces. The “guides and sensors” framing from Birgitta Böckeler’s post on harness engineering was used in multiple sessions: guides, the context you feed an agent to steer it, and sensors, the deterministic checks that tell you whether it worked. The point of multiple harness elements is that these are separate, reusable things you combine differently depending on what you’re asking the agent to do.
What struck me is that this isn’t hand-waving about “good tooling.” People had numbers. In one side-by-side, running an agent inside a proper harness rather than letting it work unguided cut token usage by roughly a factor of four and made the output noticeably more consistent, because the harness narrows the space of tools and standards the agent has to consider. Someone else compared three ways of getting an agent to clean up code: a general instruction to “improve quality” fixed real but modest amounts; adding a linter and a general code-smell skill did better but still left more than half the known problems untouched; and a tool that took each specific signal from the linter and turned it into a specific, named refactoring instruction cleared around ninety percent. The lesson people drew was that today’s models often won’t infer which structural fix applies unless you make it explicit, because the training data rarely names refactoring as a discrete operation. Mathias Verraes describes a similar pipeline from his own work, and Abby Bangser’s account of the event has the same finding from the other side: teams that invest in cleaner module boundaries get measurably better token use and better results out of their agents.
There’s a “how” question lurking in all of this that I want to call out, because it connects two things that look separate. Does it matter how an agent arrives at a result, as long as the result checks out? The hands-on TDD session made a case that it does, and for a reason I found persuasive. The things that make code trustworthy to a human reviewer, clear names, small steps, explicit inputs, the simplest implementation that works, turn out to be roughly the same things that let an agent produce and then maintain that code reliably. The discipline transfers. So “how it’s built” isn’t a separate concern from “can I trust it.” The how is a good part of what makes the result checkable in the first place.
Which leaves one loose thread that I’m going to leave that way for now. If the rigour has moved to the objective and to the checks around it, do we still need to understand what the agent actually built?
Many of the participants framed the trust problem as being about the agent: it has no stake, it isn’t accountable, it doesn’t have your back the way a colleague does. Sam made an interesting move against that worry, arguing that the missing stake stops mattering once you stop handing the agent tasks and start handing it an objective it can test itself against. That makes sense to me. But it sidesteps the thing I actually worry about, which isn’t the agent’s stake but our own.
When we let an agent make decisions we don’t fully follow, we’re not just trusting the agent. We’re accepting that our own understanding of the system is getting thinner. Do that enough and you’ve built something nobody on the team can actually reason about, and at that point the confidence problem isn’t about whether the agent is any good. It’s that you no longer have the understanding you’d need to know either way. That’s the thread I want to pull on properly in a future post, so I’ll set it down here rather than resolve it.
The sharp edge
Everything so far has been pretty abstract. Production is where it stops being abstract, because the cost of a wrong call is immediate and has consequences that may be hard to undo. So it’s the place where “how much do we let the agent decide” gets asked most sharply, and where I found the disagreement most useful.
There was broad agreement on the current state of things: agents are genuinely good at noticing that something is wrong and working out what, and not yet trusted to decide what to do about it. Several people described setups where a telemetry alert triggers an agent automatically, the agent runs a structured set of diagnostic steps, and then it hands a human an organised summary of what’s likely going on. And then it stops without taking any action itself.
Where people split was on how far past that line you can go. Some were bullish that agents can, at least in principle, be trusted to carry out remediation for some class of problems. Others thought their proper role is to gather the picture and put a set of possible fixes in front of a human to choose from. And a third group were uncomfortable even with that middle position, because the moment an agent proposes a remediation, it has anchored the human on that solution..
That last worry has a mechanism behind it that came up in more than one session, and it’s worth stating plainly. These models tend toward telling you what you want to hear. A confident agent reporting that things are fine, or that this is the fix, is not the same as things being fine, and people have been burned by that. That’s a large part of why the room was reluctant to hand over the decision itself, as opposed to the diagnosis.
Here’s where I come down, for what it’s worth, and it’s less tidy than either camp. I don’t think there’s a fixed line. We already let automation remediate production problems without asking anyone: automated restarts, autoscaling, that sort of thing. One example that stuck with me was a managed database that expands its own disk capacity, a thing that has quietly worked for years without incident, and worked precisely because a cloud provider spent real effort building a narrow, reliable, well-understood mechanism around that one action. Nobody loses sleep over letting the machines decide that.
So the question isn’t whether agents get to remediate. It’s which problems, and how tightly bounded the action is. Between “known problem, known fix, safe to automate” and “needs a human,” there’s a band of problems we can anticipate but whose fix needs more judgement than a fixed script can carry. That band is where an agent, kept on a short enough leash, can start to be useful. And I think that band widens over time, as we get better at building the leash. What’s out of reach now moves into reach in a few months, not because the models improve (although they may), but because we learn how to guide and constrain them well enough to trust them with a little more responsibility.
Which is why my instinct here is conservative in a specific way. Not “keep agents out of production,” but: don’t point an agent at your production system and give it open-ended authority to watch for trouble and fix whatever it finds. Pick specific situations you understand well, give the agent a narrow set of actions it’s allowed to take, have it act fast where that’s safe, and feed everything it does back to the humans who still own the underlying fix. Widen the range as you earn the right to.
It’s worth noticing what the operations session actually did with all this, though, because it matters for what comes next. Faced with high stakes and irreversible mistakes, they kept the agent’s remit narrow: diagnose, yes; decide, no. That was the right call for that context. But hold onto the word narrow, because I sat in another session where a team had made almost the opposite choice, and were also right.
The point is that it varies
I said to hold onto the word narrow. Here’s the other session.
One participant described their own workflow as leaning hard into agent autonomy. A human approves a plan up front, an agent goes away and does the work, and the human reviews the result at the end. Not the steps in between, the result. That’s a wide remit: design, build, and the path toward release, mostly handled by the agent, with a person at each end and not much in the middle. This is the opposite of the operations room’s diagnose-but-don’t-decide.
And it works for that person’s team. I’ve used their product and experienced the quality of their process first-hand. But the reason it worked is the interesting part. Their product is a tool for engineers, and is particularly amenable to understanding whether it works. Clear inputs, clear outputs, a strong notion of “correct” that you can check with tests. They can hand the agent a wide remit because they have a cheap, trustworthy way to know whether the result was good. The person who said this was clear-eyed about it, noting that the same confidence would be much harder to justify writing, say, regulated payroll software, where a subtle error is expensive and “correct” is a great deal harder to pin down.
Put the two rooms side by side and the disagreement mostly dissolves. The operations people kept the agent on a short leash because the cost of being wrong was high and a solid check on “is this remediation right” was expensive to come by. The engineering-tool person let the agent off the leash because being wrong was cheap to detect and cheap to fix. Neither was being braver or more cautious than the other by temperament. They’d read their situations correctly and set the controls accordingly.
That’s the thing I kept seeing once I noticed it. How big a piece of work you hand an agent, how much of the job you let it own, how much preparation you do before you delegate, how closely and at what level you check the result, what you build around it to keep it in bounds: these aren’t independent knobs you set by preference. They move together, and they move with the situation. A cheap, reliable check on the outcome buys you a wider remit. An expensive check, or a high cost of being wrong, forces a narrower one. Much of what sounded like principled disagreement in the room was people with different settings, correctly chosen for different contexts.
Which is why “it depends on context” isn’t the cop-out it usually is. It’s the actual finding. Ivett Ördög’s writeup lands in the same place on the dark-factory question: viable for some things, transactional backends, COBOL rewrites, but not an answer for everything. Sam Ruby makes a sharper version of the point about the room itself, noting that a gathering of people from large, established organizations and the consultants who advise them will naturally reach for how do we govern this, verify this, tier it by risk, long before how do we use this to put someone else out of business. Even which questions felt urgent was a product of who was in the room.
And the frontier doesn’t move evenly. One of the conversations that most tested my own assumptions was about building knowledge graphs. The received wisdom is that the relationships in a knowledge graph are exactly the sort of tacit, hard-won domain knowledge that a human expert has to put in by hand. I challenged that directly, describing knowledge graphs built as a near-incidental byproduct of pointing agents at the right signals. It’s genuinely unsettled, and Gregory Herlein’s notes capture the same open question from the event, along with a nice observation that a graph built this way turned out to be about as accurate as vector search while being faster and cheaper. The point isn’t who’s right. It’s that the boundary of what you can safely delegate isn’t a smooth line creeping forward. It jumps ahead in places nobody expected, which is exactly why a fixed rule about “what agents can be trusted with” goes stale so fast.
Where this leaves me
So the boundary of agent autonomy moves, it moves unevenly, and where it sits for you depends on things specific to your situation: how expensive it is to check the work, how costly a mistake would be, how much you’ve invested in the preparation and the checks and the harness around the agent. A fixed rule about what agents can be trusted with is going to be wrong for someone, and stale for everyone soon enough.
That’s an awkward place to stop, and I don’t think the answer is a better rule. I think it’s a better way of describing the thing you’re actually deciding. Underneath all of these sessions, the operations debate, the wide-remit team, the dark-factory spectrum, the argument about who’s allowed to steer the model, people were making the same handful of choices over and over about a single thing: the unit of work they were prepared to hand to an agent. How big it is. How much of the job it covers. What you do to get it ready to hand over. How you check what comes back. What you put around the agent to keep it inside the lines. Different rooms set those differently, but they were setting the same controls.
And I think that reframes the confidence problem I left hanging earlier. The worry was that delegating work we don’t follow leaves us understanding our own systems less and less. The way out isn’t to refuse to delegate, or to insist on reading everything the agent produces. It’s to let the level at which you need to understand the system rise as you hand off more of the detail: to understand it at the level of intent and boundaries and behaviour, while an agent works below that line, and to keep the ability to lift the lid and check when you need to. You give up understanding every line. You don’t give up understanding the system. Done well, that’s not a loss of control. It’s control at a more useful altitude.
Getting that right, deciding how far to hand off and what to keep, is not something I think anyone does well by instinct yet, including me. So in an upcoming post I’m going to try to make it concrete: a way of looking at an agentic workflow in terms of that unit of work, the span it covers, the preparation, the review, and the harness around it, so you can see where you currently sit and reason about where to move next, instead of arguing about it in the abstract. That’s the tool I wish we’d had in the room.
Photo by redcharlie on Unsplash
