The smart/dumb contract

This is the intellectual core of harness-mini: keep every agent's context in the smart zone at all times.

Definition — a clean-context budget, with occupancy as the cheap proxy

“Smart” and “dumb” name a budget for keeping context clean. 40% occupancy is the operating trigger we measure — but, as the benchmark below shows, what fills the window matters more than how full it is. Treat 40% as a conservative checkpoint default, not a law.

• Smart zone — below 40% occupancy (default trigger). Comfortable headroom to reason, follow instructions, and write a clean handoff.
• Dumb zone — at or above 40%. Not a measured cliff — a checkpoint-now line drawn early so quality never gets the chance to slide.

The threshold is configurable (HARNESS_CTX_THRESHOLD, default 40); 40 is the deliberate default. Every agent — main and sub — is responsible for keeping itself smart.

Tested: the CIB benchmark

The 40% line was, for a long time, a heuristic backed by a citation. In v0.9.0 the harness put it on trial. We built CIB — a context-intelligence benchmark (bench/cib/) — and ran it live over OpenRouter on three models: gpt-4o-mini, claude-haiku-4.5, and qwen-2.5-7b-instruct — the exact model from the paper (arXiv:2601.15300) that motivated the line. Occupancy is measured from the real usage.prompt_tokens each response reports.

The design is the whole point. Earlier “context cliff” results confound length with difficulty — longer natural documents are also harder documents. CIB holds the probe identical and varies only the filler, so any decline is attributable to occupancy alone.

Result 1 — raw occupancy showed no 40% cliff

Under the controlled design, no model exhibited the paper's 40–50% intelligence cliff— including the paper's own model. Frontier models held retrieval quality smart all the way to ~78–80% occupancy.

Reasoning was probed too — a controlled multi-hop replication on Qwen2.5-7B, calibrated to a mid-range baseline and run to n=25 per bucket (a low-N pass produced a false cliff that n=25 erased). Pass-rate by occupancy from 11% → 82% stayed flat within a ±15 confidence interval — the most-filled bucket scored as well as the emptiest.

Result 2 — interference is the real driver, not occupancy

The decisive experiment used the paper's actual task (HotpotQA multi-hop) and metric (token-F1). The same 20 questions were asked at every occupancy bucket — difficulty held constant — and the window was padded two ways:

• filler arm — irrelevant repeated text: occupancy ↑, interference flat.
• distractor arm — competing, related Wikipedia paragraphs: occupancy ↑, interference ↑.

At the matched ~10% baseline the two arms are identical (52.8 vs 53.8). As the window fills, the filler arm stays flat while the distractor arm declines, and the gap widens to ~20 F1 points — a gradual decline, with no sharp 40% step in either arm. Filling the window with irrelevant text is tolerated; filling it with competing, related content is what costs quality.

The verdict

Raw context occupancy alone shows no 40% intelligence cliff under a controlled design. What degrades quality is interference — what fills the window, not how full it is.

So the line's real job is to bound interference, and the mechanisms below — fan-out distillation and progressive disclosure — earn their keep by keeping competing content out. Occupancy is just the proxy we can cheaply measure. 40% stays as a conservative engineering default(checkpoint early — cheap insurance), not an empirically-pinned constant; the “a paper proved 40%” support is withdrawn. This is recorded as assumption A1 in docs/assumptions.md.

Honest limits.CIB's filler is synthetic and may be more “ignorable” than dense natural prose; the confound test is one model (Qwen2.5-7B), one dataset, with wide (±~20) confidence intervals, and is not yet run on a frontier model. So a pure-occupancy effect on dense natural content is not ruled out— only that a controlled probe on the paper's own model does not show the advertised cliff. Full method, data, and the self-contained HTML reports live in bench/cib/results/FINDINGS.md.

How the line is held (in priority order)

1. Sub-agent fan-out — the load-bearing mechanism

The only mechanism that does not depend on an agent honestly measuring itself. The main agent stays smart by delegating every heavy or dirty operation — broad searches, large-file reads, log scans — to a disposable explorer. The explorer is allowed to fill its own window to ~90%, then returns a distillate (a few hundred tokens) and dies. The caller absorbs only the distillate. Structural, therefore reliable.

Rule of thumb: any operation that would pull in more than ~2k tokens → delegate it.

2. 40% = the checkpoint-and-reset trigger (not a hard wall)

Crossing 40% does not mean “stop blindly.” It means: checkpoint now, while you are still sharp. Write .trace/checkpoints/<plan>-<seq>.md, update the active exec-plan, then reset to a fresh session that bootstraps from that artifact. Waiting until 90% is fatal — the handoff itself would be written by a degraded agent.

3. Measurement — best-effort, to see drift

bin/ctx.sh <used_tokens> [window] prints N%and exits 2 at/over threshold. You cannot portably read a model's internal token counter, so this is an estimate. On Claude Code the opt-in bin/ctx-hook.sh PostToolUse adapter does better — it records a ctx_pct sample after each tool call and nudges you to checkpoint once you cross the line. harness.sh report then aggregates those samples into a context trend (sample count, max %, crossings) — enough to watch drift in .trace/, not a hard interrupt.

4. Progressive disclosure keeps the baseline low

AGENTS.md is a ~100-line map of pointers, never an encyclopedia. Start near-empty; pull a file only when you need it. This buys the most headroom for the least effort.

5. One bounded task per session

A session scoped to a single feature / plan-step never needs to hold everything, so it naturally stays under budget.

Enforcement style

Behavioral + structural, not a hard runtime kill: agents follow this protocol and the explorer firewall does the heavy lifting. On Claude Code, the opt-in ctx-hook.sh adds a real-usage tripwire on top.

Entropy: smart context decays

Smart context is append-rarely, prune-aggressively. Over time, once-useful “always-loaded” facts rot. The gardener agent periodically demotes stale smart context back into dumb (on-demand) docs and flags drift — the garbage collector that keeps the smart zone smart.