Agent Harness
Developer Tooling / Automation
Outcomes
Every scheduled job that submits an application or sends mail under my name now runs from the synced worktree. The five-day stale-branch failure cannot recur in the same shape, because the branch is no longer an input.
Build gates, verification enforcement and destructive-git refusals run as harness hooks rather than as instructions in a prompt, which is the only version that holds when I am not the one reading it.
The reply-fetch path has failed 86 times running. It cannot read the macOS Keychain under launchd, catches its own error, writes a one-line summary and exits 0. In launchd and in the digest it looks healthy. This is filed, diagnosed and not yet fixed.
Overview
The tooling layer underneath a multi-project engineering workspace that runs on Claude Code. An MCP server exposing a typed playbook registry whose contract refuses irreversible side effects, 39 reusable agent skills, and 38 lifecycle hooks across 11 events that gate builds, enforce verification and block destructive git operations before they execute.
Six of those jobs run unattended on launchd timers and act outward, submitting applications and sending mail under my name. That is the part that changed how I build. An agent you are watching is a demo. An agent that fires at 6am on a 20 minute interval is infrastructure, and infrastructure fails in the one way a demo never does: quietly, while reporting success.
Challenge
For five days in July one of these daemons emitted a wrong digest every single morning, 62 false entries a day, and nothing anywhere reported a problem. The job succeeded. launchd recorded a clean exit. The logs were clean. It had simply been running a stale feature branch, because its working directory was a live developer checkout and I had left that checkout parked on the branch. I found it by accident, days later, when an unrelated `git checkout main` happened to fix it. Nothing in the system was capable of noticing, and the failure generalises past this one job: a scheduled task does not run your code. It runs whatever is checked out at the moment it fires.
Tech Stack
Architecture
Key Features
The Job Runs Pinned Code
Every scheduled daemon now executes from a dedicated worktree on its own branch, fetched and hard reset to origin/main before each run, with the resulting HEAD compared against origin/main before anything is allowed to proceed. The live checkout can be dirty, mid-rebase, on any branch at all. These jobs never read it again.
A Guard That Cannot Silence What It Guards
The first version of this asserted a clean tree and aborted otherwise. That is the obvious fix and it was wrong: it meant three daemons silently did not run whenever I had work in progress checked out, which for a job whose entire purpose is to act is the same failure wearing a different hat. Syncing a separate worktree removes the conflict instead of arbitrating it.
Failures That Refuse Instead Of Guessing
Every abort path in the runner raises an alert and exits non-zero rather than continuing on a best guess. A missing worktree, a hand-checked-out branch, a fetch that failed, a post-reset SHA that does not match: each one refuses the run and says which one it was. The guard is the one component in the system that is loud on purpose.
Code
The pin, in full
CURRENT_BRANCH="$(git branch --show-current || true)"
if [ "$CURRENT_BRANCH" != "$PINNED_BRANCH" ]; then
fail "worktree is on '${CURRENT_BRANCH}', expected '${PINNED_BRANCH}'."
fi
git fetch origin main --quiet || fail "git fetch origin main failed."
git reset --hard origin/main -q || fail "git reset --hard origin/main failed."
WORKTREE_SHA="$(git rev-parse HEAD)"
ORIGIN_SHA="$(git rev-parse origin/main)"
if [ "$WORKTREE_SHA" != "$ORIGIN_SHA" ]; then
fail "post-reset HEAD (${WORKTREE_SHA}) != origin/main (${ORIGIN_SHA})."
fiThe last check looks redundant after a hard reset, and it is the one I would keep if I could only keep one. It is the difference between believing the reset worked and knowing which commit is about to run. `fail` alerts and exits non-zero; there is no branch through this block that continues on a maybe.
Lessons & tradeoffs
I spent most of this project fixing the same bug wearing different clothes: something did not happen, and everything reported success. A stale branch exits 0. A daemon that never ran exits 0 by never being asked. A caught exception written to a summary line exits 0. Automating work you do not watch is much less about making agents smarter and much more about refusing to let 'no error' be evidence of anything. The pin is only twelve lines. Believing the twelve lines instead of checking the SHA afterwards is what the five days cost.
A launchd job with its working directory set to a live checkout runs whatever branch that checkout is on. Mine sat on a feature branch for five days and shipped a wrong digest every morning, with a clean exit code and clean logs each time.
I stopped asserting anything about the developer checkout and gave the daemons their own worktree, fetched and hard reset to origin/main before every run, with the post-reset SHA compared against origin/main before any work starts.
Pin the code, do not inspect the environment. An assertion tells you the state was wrong after you are already in it; a sync makes the wrong state unreachable. And 'it ran successfully' answers a different question from 'it ran the right code', which I had been treating as the same question for months.
My first fix aborted the run whenever the tree was dirty. It worked exactly as designed, and it meant three daemons quietly stopped running every time I had uncommitted work, which was most of the time.
I moved them onto the syncing runner so the live tree's state is irrelevant, and reserved hard refusal for conditions that genuinely make the run unsafe rather than merely untidy.
For a job whose purpose is to act, not running is a failure too, so a guard that fails closed on a common developer state has invented a second outage while fixing the first. Guards need to be judged on both of their exits, not just the one you were thinking about.
One daemon has now failed to fetch replies 86 times in a row. It shells out to the macOS Keychain, which is unreachable under launchd, catches the failure, writes it into a summary string and exits 0. Every health surface I have says it is fine.
I filed and diagnosed it rather than quietly patching it, and I am leaving it stated here while it is still open, because it is the exact class of defect the rest of this system was built to catch and it got through anyway.
A harness that pins the code still cannot tell you the code did anything. Exit 0 and quiet logs are compatible with total inactivity, and the fix is not more logging, it is a check that asserts the work happened. I would rather publish the open one than imply the harness is finished.