Last time I argued that the coordinating layer of an agent system should stay deliberately dumb — slow, discrete, symbolic — while the perceptual and temporal work gets pushed down into specialized submodels. The obvious follow-up question, and the one I got asked most, was: fine, so what does that actually look like? Isn't a dumb orchestrator just a control loop with tool calls, which everyone already has?
That's the right challenge, and answering "draw the box diagram" doesn't meet it — because the diagram does look like an agent loop. The difference isn't the shape. It's the contract at the seam. Dumb-by-design is a set of rules you hold to about what crosses the boundary, and every one of them is a rule most real systems quietly break. Here are the five that matter.
1. The orchestrator holds no perceptual state. It never touches the waveform, the pixels, the raw stream — only summaries handed up by submodels. This sounds obvious and is violated constantly, because there's always one feature that would be so easy if the coordinator just peeked at the raw signal directly, just this once. Don't. The moment the orchestrator reads raw input, the abstraction it's supposed to reason over stops being the real interface, and you've built a monolith with extra latency. The discipline is the whole point: if the summary isn't rich enough, you fix the submodel's report, not reach around it.
2. The interface is a narrow typed vocabulary, not tensors. What crosses the seam should be things the orchestrator can log, branch on, and reason about in symbols — speaker_changed, confidence: 0.6, out_of_vocabulary, silence > 800ms. Not a 4096-dimensional embedding. Scalars are fine; the test isn't dimensionality, it's this: could a human reading the event log have made the same decision the orchestrator made? If yes, the seam is symbolic. If the decision needed information no one can read off the log, something dense snuck across, and this is the real dividing line between this design and MoE or any end-to-end system — if the handoff is a dense vector, the two halves are one differentiable blob and every claim I made about legibility evaporates.
3. The orchestrator is enumerable. Meaning: you can list its states and transitions exhaustively. You should be able to print the coordination logic — the states, the transitions, the branch it just took — and have it fit on a screen. Memory and history can be large; that's fine, they're data the orchestrator reads, not logic it runs. What has to stay small is the decision-making itself. "Dumb" means legible, and legible is the payoff you're buying with the lossy seam from post one — you accepted throwing information away, so collect the return. If you can't enumerate the logic, you don't have a dumb orchestrator, you have a small opaque model doing coordination, which is the thing you were trying to avoid. And notice the uncomfortable consequence: if your orchestrator is an LLM, you've already broken this rule. The "agent loop everyone already has" almost always puts a language model in the coordinator's seat, and a prompt loop is many things, but enumerable is not one of them. That, more than any diagram, is what separates this architecture from the one you're probably running.
4. Submodels are swappable without retraining the coordinator. The test of whether your boundary is real rather than decorative: replace the audio submodel with a different one that emits the same vocabulary, and the orchestrator shouldn't notice. If swapping a submodel forces you to touch the coordinator, they were entangled and the seam was fiction. This is also what makes the architecture improvable — each piece has its own lifecycle instead of one monolith you retrain wholesale.
5. The orchestrator is allowed to be wrong cheaply. This one is less a rule you hold than the dividend the first four pay. Because the coordinator is simple and legible, when it makes a bad call you can see exactly which summary misled it and which branch fired. That's the concession from the first post turned into a feature: yes, the seam is lossy and the coordinator can be confidently wrong about an abstraction it can't see behind — but when that happens, it's a five-line diff, not an archaeological dig through a billion weights.
One honest caveat before the example, because rule 4 is easy to overclaim. Swappability holds within a fixed vocabulary. The vocabulary itself is the one artifact both sides depend on, and changing it — adding an event type the coordinator has never heard of, splitting one event into two — is the one legitimately entangling operation in the whole design. New symbols mean new branches. Systems built this way don't die of bad substitutions; they die of vocabulary churn, where every schema revision quietly touches both sides of the seam. I don't have a clean answer to that, other than to say the vocabulary deserves the same reluctance to change that you'd give a public API, because that's exactly what it is: the API between the smart parts and the dumb one. Design it like something you'll be stuck with, and treat every extension as the expensive operation it actually is.
Make it concrete. Say you're building a live voice agent — something that listens continuously and acts on what it hears. The submodels do the hard part: one turns audio into text, one tracks who's speaking, one flags when the acoustic scene changes. Each runs in its own regime, on the continuous signal, at its own clock. What reaches the orchestrator is a thin stream of typed events: utterance_complete, speaker: B, confidence: 0.4, wake_phrase_detected. The orchestrator's entire job is a boring state machine over that stream. Boring as in: states IDLE, LISTENING, CONFIRMING, ACTING, WAITING, with transitions triggered only by the typed events above. That's rule 3 demonstrated rather than asserted — the whole coordination logic just fit in a sentence. The orchestrator never hears a single sample of audio. And that's exactly why you can read its logs and know precisely why it did what it did, swap the transcription model next quarter without touching the logic, and fix a misfire by editing a transition rather than fine-tuning anything.
Contrast that with the version most people are building: one end-to-end model ingesting the audio stream directly, inferring turn-taking through attention, deciding whether to respond — all in the same undifferentiated forward pass. When a door slams and it answers a question nobody asked, there's no event to inspect and no transition to fix. The failure is somewhere in the weights, and "somewhere in the weights" is not a place you can go.
Notice what the orchestrator is not doing in that example. It isn't clever. It holds no embeddings, runs no attention, has no parameters worth the name. All the intelligence lives below the seam, in the parts built for exactly one job. The coordinator is dumb on purpose, and the system is better for it — more legible, more swappable, more debuggable — precisely because nothing smart is happening at the top.
That's the shape. Not a new diagram — a contract. The five rules are just ways of saying the same thing: keep the intelligence below the seam, and keep the seam made of symbols you chose. A symbol you named will always be worth more than an activation you hope means something.
Comments
Post a Comment