Agentic Harness: What It Costs to Build One
Part one ended with five things I want from any agentic harness. This is what each of the five actually costs to build, which part of each one fails quietly, and how to check whether the harness you are using has them.
Why an agent needs a harness ended with a few things I want from any harness:
- Real isolation.
- A visible difference between done and gave up.
- Permissions built for nobody watching.
- A transcript that survives the run.
- Room to run more than one job at once.
1. Real isolation
The wall is the cheap part. Starting a container per task is an afternoon.
What costs is what the wall does to everything around it. Once the work happens inside a box you are not sitting in, the job cannot be the terminal any more. It has to outlive the terminal that declared it, which means the job is a record somewhere rather than a process on your laptop, and something has to write that record, start it, notice when it ended, and keep what it produced. The moment you can close the lid and walk away, you own a life cycle you did not own before.
Then there is the second wall, the one inside. A container decides where the work runs. It says nothing about what the worker is allowed to do. That is a separate thing to model, and the useful shape is a role: which model runs this job, what it is allowed to read, and which verbs it holds. Verbs are where it earns its keep. A role that ships work cannot declare work. A role that coordinates does not hold the editing verbs.
An agent that can decide what to do, do it, and then declare it done has no boundary inside it, however good the box around it is.
Isolation is a role, not just a box
The box decides where the work runs. The role decides what the worker is allowed to do.
The cost: a job record with a life cycle, and a verb list written down before the first run. The test is the one from part one. Name a job and say precisely what it can and cannot touch. If the answer is a paragraph rather than a list, that is not isolation.
2. A visible difference between done and gave up
A model's natural output is prose, so a harness that does nothing in particular gets prose. "I have updated the handler and it should now work" is not an outcome. It is an essay about one, and it hands the decision to whoever reads it, which is the job you were trying to give away.
The fix is unglamorous and it is not the same thing as asking for a better summary. First enumerate the outcomes a job may end in and then match the ending against that set exactly. Anything matching nothing in the set is a failure, not free text to be interpreted later. And "gave up" has to be one of the outcomes, with a reason attached.
That gets you an honest report of the run. It still does not tell you the work exists. A green check is a claim, and the claim is only ever about the steps. I have watched a pipeline run every step it was given, report success, and ship nothing at all, because the identity it assumes could not create the thing it was asked to create. Nothing threw. Nothing arrived.
Two questions, not one
Ask the session whether the outcome is there. Do not ask the run whether it finished.
The cost: a closed set of endings the harness enforces, and a check at the end of each job that asks whether the work exists.
3. Permissions built for nobody watching
When a person is at the keyboard, permission is a question asked at the exact moment it matters, by something that knows precisely what is about to happen. Take the person away and that question has no answer. The session does not fail, which would be fine and expected. Instead it just waits.
There are two ways to be wrong about this step. Guess too broad and you have quietly handed back the isolation you paid for in the first section. Guess too narrow and the run stops, or worse, carries on and delivers nothing, which is the pipeline again: a grant made in advance, too small, and not one part of the system able to say so out loud.
Roles are the cheapest place I have found to put this guess, because a grant attached to a role is made once for a kind of work rather than once per run, and somebody can read the list and disagree with it. It is still a guess. It is just a reviewable one, and a narrow guess is only safe if the previous section is already built, so that refusal comes back as an outcome you can see rather than as an empty result.
4. A transcript that survives the run
There is a difference worth being pedantic about. What a run says about itself is testimony. What the run actually did, the commands, the outputs, the diffs, the refusals, is evidence. You want the evidence, so the process can be scrutinised and challenged.
The cost: isolation says the box is disposable and should be thrown away the moment the task ends. Evidence says nothing readable inside that box may be lost. So you pay for the wall, and then you pay a second time to lift everything worth reading back over it before it goes.
5. Room to run more than one job at once
Running many jobs at once is not just parallelising the flow described up to here. It means creating admissions, claims and guard checks.
Admissions: something has to decide whether a job starts now, because capacity is finite: model quota, containers, machines, money. With no admission decision the answer is that everything starts at once, and everything starting at once behaves remarkably like nothing starting at all.
Claims: two workers must not build the same thing, so a piece of work has to be claimed before it is started, the claim has to be visible to every other worker, and it has to be released when the work ends or expire on its own if the worker disappears. Get this wrong and the second worker's output is not merely wasted. It conflicts with the first, and now somebody is reconciling two attempts at one job.
Guard checks: something watching if the system has stopped moving, checking nothing crashes when work is added or stops flowing.
What running many at once actually needs
Nothing throws an error when the queue stops draining. You have to go and ask.
Guard checks is the cheapest thing on this entire list and the easiest to skip. Work waiting and nothing running is one query away, it needs no new component. Write the query, run it on a schedule, treat the answer as an alarm.
Parallel work is also where the harness starts carrying weight. With one run, watched, you can read the transcript yourself and tell if work has been done by looking at it. With several, unwatched jobs, judgement can drift and the harness can become the bottleneck if it is not set up properly.
What I have not solved
The initial refinement: how can we tell if the system understands contextually what we have in mind? This is a solved problem for narrow scope and small tasks, but when the work becomes broader, quality takes a hit and the jobs start to drift. My next attempt would be to introduce cadence and work review in agile fashion, so the system is always honest and actual value is delivered and measured.
This is the second post in what I want to make a short series on building an agentic harness. Let me know if you have any more features that you think would be beneficial in a harness. Happy coding.
Questions about the build, or building something similar? Say hello. I read every message.