Every week someone wraps a model in a harness, hangs four tools off it and presents it as “the company’s agentic operating system”. No identity, no authority boundary, no journal, no budget and not a single measurement of scale. This article tries to put the bar where it belongs: which parts you need, how they talk to each other, which ones cannot go down, and where, however far we dim the lights, you still need a person.
Listen in my own voice · 29 min
Speed:
I’ve spent a good while with my hands deep in building one of these systems. And the further I get, the less patience I have for the label.
The problem isn’t that “operating system” sounds grand. The problem is that it’s a promise. An operating system doesn’t do your work: it makes work possible, safe and progressively cheaper. It shares out the CPU, isolates processes, decides who may open which file and keeps a record of what happened. If you claim those two words, you owe those services. And most of what I see under that label doesn’t provide them: they are very clever applications with nothing underneath.
A few weeks ago I wrote that, in benchmarks, the harness sets the score. In an organisation something similar happens, with bigger consequences. The harness decides how an agent works: what context it sees, which tools it has, how many turns the loop takes. What it doesn’t decide is whether that agent should be allowed to do what it does, on whose behalf, with which data, or what it’s costing anyone. That’s another layer. This one.
The missing row
Classic operating systems have users, processes and permissions. What they don’t have is a construct for an agent: a principal that acts with its own, narrowed permissions while still answering to a specific human. A process inherits the permissions of whoever launches it. An agent shouldn’t. That missing row in the table is, quite literally, the reason an agentic operating system exists.
If you take the metaphor seriously, almost every part finds its counterpart:
- The kernel is a library of pure functions that returns every verdict: expiry, budget, scope, data egress, merge. It has no connection and no clock, so any verdict can be replayed later.
- System calls are the governed acts —read, search, write, egress— and they all cross a broker. The endpoint is called, without any shame,
/syscall. - Processes are missions: who acts, for what, with what budget and until when. They form parent and child chains.
- Permissions are signed grants, with scope, expiry and budget. A child always holds less than its parent.
- sudo is an approval gate: a named person decides, and only with evidence in front of them.
- The file system is the organisation itself, written in Markdown inside git.
- The audit log is a journal with a single writer and no deletion.
And where the analogy breaks, you learn as much as where it fits:
- The kernel is not a privileged runtime. It’s an ordinary library you import. Its power doesn’t come from running in a special ring, but from being the only place where rules live, and from the broker being the only door.
- The operating system doesn’t schedule the work. Runtime, scheduler and shell are commodities. You don’t rebuild them: you wrap them in rules.
- The “file system” is policy, not storage. Nothing in it executes. A file grants nothing until the broker loads the manifest generated from it.
- Every chain ends in a human. No classic operating system has an equivalent of this law: every action can be traced back to a person who answers for it.
A dark factory, but not entirely
The expression isn’t mine. Early this year, Dan Shapiro proposed a five-level ladder of autonomy in AI-assisted programming, modelled on the self-driving one, and on the top rung he put the dark software factory. The image comes from the Fanuc factory where robots build robots: it’s dark because, in Shapiro’s words, it’s a place where humans “are neither needed nor welcome”.
I like the image. I don’t like the conclusion.
A beehive works in the dark. Nobody checks it cell by cell, and honey comes out. But a hive isn’t a system without control: it’s a system where control lives in the structure. In the geometry of the cells, in the pheromones, in the guard bees at the entrance smelling every bee that comes in. Darkness isn’t the absence of oversight; it’s oversight that doesn’t need light.
That’s the thesis of this article: a dark software factory is not a factory without light. It’s a factory where the light comes on exactly where it’s needed, and the structure holds up the rest. And that structure is the operating system.
With that, I can give the definition I’ve been refining for months:
An agentic operating system is the layer that makes agent work possible, safe and progressively cheaper, without doing that work. It has three non-negotiable parts: the organisation’s definition of itself, kept as versioned text; one deterministic authority boundary that every agent action crosses; and a journal with a single writer, so every act can be traced to an accountable human and to the rule that allowed it.
The dark factory is what you can build on top once those three things exist. Without them, what you have is an assembly line wearing a blindfold.
The anatomy
The architecture I’m proposing is fully distributed. At the centre sits a Kubernetes cluster that hosts the runners and the pods doing the heavy lifting. Around it, a set of planes, each with its own verb, that decide what may happen inside the cluster and keep a record of what does.
Since I find it hard to think about architecture without thinking about bodies, I’ll tell it this way:
- The interaction plane is the skin: it senses, receives, proposes. It moves nothing by itself.
- The platform plane is the skeleton: you don’t design it from scratch, you inherit it, and everything else rests on it.
- The authority plane is the spinal cord: it decides by reflex, without asking the brain, and always the same way.
- The execution plane is the muscle: it does the heavy work, but only moves when the signal arrives.
- The knowledge plane is the immune system: it tells self from non-self and expels what it doesn’t recognise.
Two rules of direction hold the whole drawing up, and they’re worth learning before going into detail:
- Authority attenuates downward. It’s fixed when a grant is issued, and delegation can only narrow it. Never widen it.
- Observation accumulates upward. What a chain of agents has seen only grows, and it’s re-read at every checkpoint, not frozen when the grant is issued.
The chapters below go from top to bottom. I explain the platform before authority on purpose: authority uses platform services —identity, secrets, cost— and it makes much more sense once you know where they come from. Knowledge comes last because you only reach it after you know who’s allowed to read it.
Each plane gets a table with three questions: what each part does, what happens if it fails, and its criticality. Criticality isn’t an opinion about how important a component is; it’s the answer to “if this goes down, does everything stop, does one part degrade, or does hardly anyone notice?”
Chapter 1. The interaction plane: the skin
Skin has millions of receptors and no muscle of its own. It senses, transmits, and that’s it. This plane is the same: this is where people come in —the intake assistant, the chat, the use-case board, the admin panel, the organisation repository, the merge request comments— and none of these components changes a record on its own.
The plane’s rule is short: it proposes, it never mutates. An agent in the chat can draft a change, but the person saves it. That even applies to their own data: if an agent offers to update your profile, it prepares the change; you’re the one who presses save. It sounds excessive until you think about what happens if a tool bound to a model can write: any text that reaches the conversation —an email, a web page, a PDF— turns into a write to a shared record. Prompt injection stops being a quality problem and becomes an integrity problem.
Two details of this plane I think get too little discussion:
- The intake can’t be filled in with guesses. The assistant interviews whoever asks for something and turns the request into a defined intent. An agent can restructure, validate or propagate what comes out of it, but it can’t fill a gap the interview left empty. A default that looks reasonable is a governance decision nobody made. That’s why “unknown” is a valid answer.
- The panel is the only critical part of the plane. Everything else can go down and the factory simply stops receiving work. The panel holds the kill switch, and the kill switch fails closed: if its state can’t be read, the factory stops.
Chapter 2. The platform plane: the skeleton
Nobody designs their own femur. You inherit it, and the craft lies in how the muscles attach to it. This plane gathers the services any serious organisation already has, which the operating system doesn’t reinvent: single sign-on, workload identity, the secret manager, the model gateway, git with its continuous integration, and traces.
What the operating system does do is wrap them in rules:
- People come in by identity domain, not by name. Nobody maintains lists of authorised people by hand: you decide which groups may use what, and the organisation’s sign-on does the rest.
- Jobs are actors with their own identity, verified with tokens signed by the platform. And a permission held by a job is never inherited by a child mission.
- No secret lives in git. A data source states the route to reach it; it never carries a token. Factory sandboxes receive the model key on a single process’s input, and the code repository token never enters a sandbox.
- Models are only built at declared sites. It sounds like an architect’s obsession until you translate it: if anyone can instantiate a model anywhere, cost, model choice and credentials end up where nobody can see them. A cost nobody counts is a cost nobody governs.
Almost everything here is critical, and that’s normal: it’s the skeleton. If the secret manager goes down, everything that reads data or calls a model stops. If the gateway goes down, there’s no inference. What’s interesting is what isn’t critical: traces. They’re for debugging, but the audit isn’t the trace, it’s the journal. Confusing the two is one of the most expensive mistakes I see.
Chapter 3. The authority plane: the spinal cord
When you touch a burning pan, your hand pulls away before your brain finds out. It’s a reflex arc: a decision that doesn’t go through the organ that thinks, always comes out the same, and arrives in time. The authority plane works exactly like that, and its motto sums up the whole article: the model proposes; a deterministic layer disposes.
The parts:
- The broker is the only door. Every governed action crosses it on behalf of a mission with a signed grant. And it has a very healthy habit: if it starts misconfigured, it refuses to start. I’ve seen enough components that “started fine and governed nothing” to consider this one of the most important decisions in the design.
- The governance kernel is the set of pure functions that return each verdict. Rules live there and only there. Copies of a rule don’t stay equal; they stay plausible, which is worse.
- Missions and grants. A mission ties the work to an intent defined in the organisation repository. If the intent doesn’t exist, the broker refuses. And a mission’s identifier isn’t a credential: acting requires the signed grant. Reading the evidence of what it did doesn’t, because a mission only its holder can audit isn’t audited.
- The control journal. One writer —the broker— and no delete permission in the database. A single writer is what turns a log into a record, rather than a collection of opinions. And here’s the detail that took me longest to understand: denials are written too, in a separate transaction. A denied call is rolled back —it leaves no observation and no charge— but the “no” gets recorded. Otherwise the journal would only hold “yeses” and nobody would notice the silence.
- Every verdict carries its policy version. So when you replay a decision months later, you can tell “the rules changed” from “the engine is wrong”.
- Approval gates. A named person decides, with the battery of checks in front of them. And there’s a deliberate asymmetry: they may refuse anything, but they may not approve red. A verdict with no checks behind it is an opinion, and “someone approved it” audits nobody.
The whole plane is critical except the gates, which are “only” high: if nobody decides, the stages that need a person wait, but nothing breaks.
Chapter 4. The execution plane: the muscle
This is where the Kubernetes cluster lives, and where the lights go out. It’s the plane that most resembles what people picture when they hear “dark factory”, and also the most closely watched.
This is how a task works:
- A runner asks the broker for its own permission before every cycle. It carries no permissions over from the previous cycle.
- It claims one sandbox per task: an isolated pod with networking closed by default.
- It hands the agent a clone of the code with no remote and no credentials. The agent can write all it wants; it can’t push anything anywhere.
- Everything that comes back from the sandbox is treated as untrusted bytes: size-bounded, strictly parsed and integrity-checked.
- The battery —the tests that go from red to green— runs in a second sandbox that holds no key at all. Without a battery, “done” is only the agent’s word.
- A draft merge request is opened.
Then comes the adversarial review, which is my favourite part. The reviewers run on a different, stronger model than the one that implemented, and a test makes sure the two never coincide. And the judge holds no pen: the reviewing model has no code repository token, and a separate poster validates its output against a strict schema before it reaches anywhere. If there are findings, the line runs at most two fix rounds on its own. Whatever remains below blocking level can go through, but with the debt in plain sight.
At the end sits the merger, which only merges if the kernel says yes: paths within an allow-list, battery and pipeline green, the exact commit the line published, and a grant for a dedicated merge permission. And here I’ll be honest: today this component can run in shadow, recording what it would have merged while a person does the real merge. It’s the sensible way to switch it on: first you look at what it would have done, then you decide whether to let it.
And the brakes, which in a dark factory aren’t optional: the panel’s kill switch, which fails closed; the hard stop, which means scaling the runners to zero and survives redeploys; and a stop phrase that, written in a comment, halts one specific card’s chain. All other comments go to the agent as quoted, untrusted input: feedback refines, it never stops.
The plane’s rule has a consequence many people find excessive: no agent holds a database driver, not even in tests. A connection is authority nobody minted, and it leaves no row in the journal.
Chapter 5. The knowledge plane: the immune system
The immune system doesn’t ask what something is before deciding. It checks whether it recognises it. If it doesn’t, it attacks. The knowledge plane works the same way: data arrives labelled, or it doesn’t arrive.
- Each source declares two things: what permission is needed to read it, and which labelling rules apply to each record after it’s fetched.
- The organisation applies the label, not the connector. If the connector did it, it would be the audited party grading its own work.
- Labels form a closed set: public, internal, client-confidential and special.
- A record that matches no rule is dropped, and the drop is written to the journal. It sounds radical, but the alternative is worse: an unlabelled record would read as public to every later check.
- Everything stored carries its label and its
source@version. Without that, a month later nobody can tell a fact a person typed from a paraphrase a model decided to keep.
And the part of the whole system I like most: observation. Picture a parent agent that reads confidential material and hands a newborn sub-agent —one that has read nothing— permission to send data out. The child is “clean”, the parent hasn’t sent anything, and the confidential data has just walked out the back door. It’s laundering by delegation. The defence is that the kernel doesn’t only look at what the acting agent has read, but at what the whole chain has seen at the moment of deciding. Observation goes up; it never comes down.
Where the light goes out: critical journeys
So far, the structure. Now comes what really decides whether a dark factory goes well or badly: how much autonomy agents get, and where.
My answer is that it’s not decided per factory, per team or per model. It’s decided per Critical User Journey (CUJ), stage by stage. Each critical journey is broken into its steps, and for each step you answer a single question: who decides here?
Take the journey that scares people most: from a request to a deliverable for a client.
- The first stages have the lights on. A person asks, and a named person defines the intent by answering five governance questions: who acts, which sources it draws on, whether data is only consulted or also stored, through which channels anything leaves, and whether there are writes. None of those answers is filled in by default. And the intake’s verdict can only be “buildable now” or “buildable behind these gates”, each gate with its owner. Nothing softer.
- The middle stages are dark: agents read sources within scope, compose the draft in a sandbox, and other machines review it. There the kernel decides, and what it needs is evidence, not eyes.
- Validation is dimmed: a person validates the result, not the process. They don’t read every step; they look at what came out.
- And delivery is always lit, because anything leaving for a client is one of the action classes that should never go dark.
Those always-lit classes are five: anything leaving for a client, economic commitments, writes to systems of record, actions affecting people, and publication. However well the battery performs, someone who answers with their name decides there.
And the underlying principle: autonomy is earned per action class and on evidence, never by configuration. Agents earn autonomy; rules earn determinism.
Here I have to admit a tension I haven’t resolved. Theory says autonomy is earned with evidence. Practice says that, to gather evidence, at some point someone has to open the autonomous mode before having all of it, as a tactical measure and with the debt written down. I think that’s unavoidable. But it’s worth calling it what it is: a decision by someone with authority, not earned autonomy. Mixing the two up is the first step towards a factory nobody knows why it runs on its own.
Before you call it an operating system, measure it
Back to the beginning, because this is where the label bothers me most. You can’t call something an organisation’s operating system if you don’t know how many people use it, how many missions it can hold at once, what a task costs, or how many unsupervised merges you’re willing to swallow in a day.
I’m not going to give you my figures, and they wouldn’t be very useful anyway: they depend on each organisation. What I can give you are the dimensions that separate a harness that works in a demo from a platform for thousands of people:
- People and reach: how many distinct groups can come in, how many people launch missions each week, how many intents are defined versus those still only candidates, and how many sources have a steward.
- Work volume: how many concurrent sandboxes the cluster can hold, how many use cases move in parallel, how big each task is, and which factory types it serves. Code, documents and data need different batteries.
- Cost: per task, per card, per day and, above all, per deliverable that is actually accepted.
- Control and risk: how many unattended merges a day, whether every “no” leaves its row, how many human validations each critical journey has, and how many hops there are between an agent and its human.
The diagram includes some example caps: 50 USD per task run, 300 runs per card, 1,000 model runs per day and 100 unattended merges per day. They’re illustrative orders of magnitude for a mid-sized organisation, not figures from any deployment. A cap is not a measurement: it’s the question the measurement has to answer. A cap nobody measures is decoration. A measurement without a cap is an invoice.
A definition to take away
If you’ve made it this far, this is the list I’d use to decide whether something deserves the name. A real agentic operating system:
- Has the organisation defined as versioned text: intents, actors, sources, rules and triggers. And nothing grants permissions until it becomes a manifest the boundary loads.
- Has a single authority boundary, and every governed action crosses it.
- Keeps rules in one place, deterministic and replayable.
- Keeps a journal with a single writer, which records denials too, along with the policy version of every verdict.
- Makes every delegation chain end in an accountable human, and authority only narrows when delegated.
- Accumulates the observation of the whole chain and re-reads it at every check, so delegation doesn’t launder data.
- Only lets classified data in. Whatever has no label is dropped and recorded.
- Separates proposing from writing: models propose, and writes go through typed proposals.
- Modulates autonomy per critical journey and per action class, on evidence and with classes that never go dark.
- Counts cost and caps it, and measures its own scale dimensions.
If your system is missing more than two, you have a harness. It may be an excellent harness, and I hope it is. But don’t call it an operating system.
The small print
Three uncomfortable points I don’t want to hide, because an article that asks for measurement can’t skip its own accounting.
None of this is a finished system yet. What I’m describing is a reference architecture with parts built, parts in shadow and parts in design. Unattended code merging, for example, makes sense to switch on after a period in shadow, not before. And the layer that would let a source be reloaded without redeploying is, for now, a drawing.
There are gaps the design admits. Checking that a mission cites a valid intent is easy. Checking that it only uses the sources that intent declared is another matter, and until that’s automated, there’s a person in the gap. I prefer a named gap to a silence.
Agent identity is still research territory. I covered it in the article on agentic identity: the standards are half-baked, and nobody covers the seam between the plane that governs and the plane that executes. A serious operating system can’t wait for it to close, but it shouldn’t pretend it’s closed either.
What I’d do on Monday
- Find every tool bound to a model that can write. Each one is a prompt injection waiting to become a write. Move them to proposals a person confirms.
- Check whether your journal keeps the “no”s. If it only keeps what was allowed, it isn’t an audit journal: it’s a photo album.
- Pick one critical journey and paint it stage by stage. Lit, dimmed or dark. If you don’t know who decides at some stage, you already have your first finding.
- Put a cap on something and measure it for a week. One dimension. The one that scares you most.
What I take away
I picture a factory like this running in the small hours. Runners asking for permission, sandboxes being born and dying, reviewers arguing with implementers in a language nobody will read, and the merger quietly noting what it would have done. Nobody’s watching. The only thing awake is the journal, writing row after row, the “no”s included.
That image gives me a bit of vertigo, and I think it should give it to all of us. But the difference between a dark factory and a factory out of control isn’t how many people are watching: it’s whether, the next morning, you can trace every act back to the person who answered for it and to the rule that allowed it. If you can, you’ve built an operating system. If you can’t, you’ve built a hive with no guard bees, and sooner or later someone raids those.
Ideas over codebases; evidence over hype.
Sources
- Shapiro, D. (January 2026). The Five Levels: from Spicy Autocomplete to the Dark Factory, discussed by Simon Willison at simonwillison.net
- The rest of the material comes from the design and construction of an agentic operating system in progress, de-identified for this article.
