The corpus is agent exhaust

A language model has no memory. It is a stateless predictor with a rolling context window, so each session starts cold (Micheal Lanham lays out the "goldfish" version of this well). To fake continuity, agent harnesses write to the workspace. They drop a TODO.md and a task list, a .plan scratchpad, a session-summary dump, a memory/ folder of notes nobody dated. I call the the whole pile agent exhaust: these are the files a harness leaves behind to simulate the memory the model does not have. It is an undefined folksonomy, markdown with links and tags that mean whatever the tool that wrote them decided they mean.

The deliberate files fragment too. One harness reads AGENTS.md, the next wants CLAUDE.md, a third expects .cursorrules. Those are at least authored on purpose, but this scratch workspace used as memory is not. The best-known version of the deliberate case is Andrej Karpathy's "LLM Wiki": raw sources plus a folder of LLM-maintained markdown pages the agent reads and rewrites. Whether your pile is a tended wiki like that or the raw scratch a harness dumps, the knowledge is real and its shape is ad hoc, so it stays trapped twhatever tool produced it. The Open Knowledge Format (OKF) exists to give that knowledge a portable shape and binder is the on-ramp. binder helps you ingest a markdown corpus, resolving its links and wikilinks and tags into OKF edges, and writes a bundle you can validate against the spec and hand to any OKF-aware agent or viewer.

I designed binder with two limits. First, a portable format does not curate itself. An agent left alone keeps dumping raw logs into the directory, and as one analysis of Karpathy's wiki put it, a neglected wiki is more dangerous than a neglected database, because stale prose still reads as authoritative. Deciding what is worth keeping is a promotion gate you have to run, whether that gate is a human reading a diff, a --strict check in CI, or an agent applying the binder skill's judgment. binder moves and stamps what you point it at; it does not decide what deserves to persist. Second, OKF is an at-rest interchange format for durable knowledge, not a runtime session-memory store. Live per-turn state (the user prefers dark mode, the meeting is at 2pm) belongs in a schema-enforced key-value or vector layer built for that job. As XTrace notes in a survey of memory tools, every tool is solving a different memory problem, and hand-maintained markdown is "temporary scaffolding" that falls apart over weeks unless something compiles and curates it. binder is your helper to manage that layer underneath the runtime engines, for the markdown harnesses persist once the state has settled into notes worthy of being called "knowledge" or a "memory."

For where OKF is going and how the surrounding tools compare, see The Open Knowledge Format Ecosystem: A Field Guide. This post is the hands-on companion on how to run binder, and which of its three surfaces to reach for and when.

Quickstart: your first bundle

Before the decision framework, run binder once end to end. Point it at a corpus you already have (an Obsidian vault, a docs folder, the accumulated exhaust of a few months of agent work) and watch it triage before it writes anything.

Preview first. A dry run reads the corpus and reports what it found without touching disk:

binder convert path/to/corpus --dry-run

The triage tells you the concept count, how many links resolved and how many did not, and which files needed frontmatter recovery. Read it before you commit. This is the moment you learn what the corpus is made of: which wikilinks point at nothing, which documents no reader will ever reach.

When the triage looks right, write the bundle. binder never mutates the source:

binder convert path/to/corpus -o path/to/bundle

Relationship extraction runs on every conversion. convert resolves standard markdown links, [[wikilinks]] and [[Target|alias]] forms, in-workspace file:// URIs, and frontmatter-ref edges, and it folds #hashtags into the tags list. Links that do not resolve stay in place and get reported rather than dropped.

Now summarize the bundle, then check it against the spec:

binder review path/to/bundle
binder validate path/to/bundle

review lists concepts by type, trust tiers, staleness, orphans, and unresolved links. validate checks the bundle against the OKF v0.2 §11 conformance rules. If both come back clean, you have a portable OKF bundle built from what was, just a bit ago, an unstructured pile of markdown.

That is the whole loop: preview, convert, review, validate. Everything below is about doing it well and about picking the right surface. For the full command reference (every flag, the JSON schema, the MCP tool schemas), see binder's user guide.

Two decisions: state & surface

Once you have run the loop once, two questions decide how you use binder day to day.

The first is the state of your knowledge.

  • You have an existing corpus. The knowledge already exists: a corporate docs site, an Obsidian vault, a wiki, agent exhaust with relative links and [[wikilinks]] and #tags. The work is ingestion, getting what you have into OKF with its relationships intact.
  • You are authoring from scratch. You write OKF concepts fresh, to spec, filling whe trust and lifecycle vocabulary as you go.

The second is the surface you drive binder through.

  • The CLI is deterministic, offline, and built for pipelines and CI gates.
  • An Agent Skill and Plugin bundle carries the judgment the CLI cannot encode and drives the CLI over its JSON output.
  • An MCP server (binder mcp) runs binder as a stdio server, exposing its additive verbs (convert, validate, review, lint, graph) to an agent runtime as MCP tools that return the same binder.report/v1 payloads as --json.

The two axes are close to independent. You can ingest an existing corpus from the CLI in a CI job, or author fresh concepts with an agent driving the skill. The rest of this post walks both states and ends with a checklist.

Ingesting a corpus you already have

This is where binder's CLI does its heaviest work, and where most agent exhaust enters OKF. A memory/ folder of notes, a workspace wiki, a directory of task-list files with [[links]] between them: these are the shapes binder ingests best. The quickstart above covered the happy path. Three practices make it hold up on a real corpus.

Give the corpus an entrypoint. A bundle wants a root index.md that links outward to its concepts. binder generates per-directory navigation, and review and lint both report orphans, concepts with no inbound or outbound edges. An orphan is a document no reader will reach by following links. Treat the orphan report as a to-do list for the corpus, not noise.

Keep cross-repo links external. When a file:// link points inside the corpus, convert rewrites it to a bundle-relative edge, and no absolute machine path leaks into the output. A link that points outside the boundary stays external and is tolerated as an advisory. The boundary defaults to the corpus root; widen it with --workspace-root when the corpus is one repo among sibling repos you want treated as internal. Links to another host, or that escape the root through .. or a symlink, stay external by design.

Gate CI on the JSON, not the prose. convert, enrich, validate, review, and lint accept --json, which wraps the report in a deterministic envelope (schema binder.report/v1) with sorted keys and a stable field set, so two runs on the same input are byte-identical. Every command maps its outcome onto a documented exit-code contract: 0 for success with advisories allowed, 1 for gating findings, 2 for a usage error, 3 for an I/O or internal error. A well-formed bundle with broken links still exits 0 by default. To make CI fail on those advisories, add --strict:

binder lint path/to/corpus --strict --json | jq '.result.findings'

lint is the pre-conversion counterpart to validate. It reads the source corpus as authored, before convert defaults anything, and reports broken links (including broken #anchors), missing titles, orphans, stale concepts, and schema violations. A missing type: is invisible in a bundle because convert supplies a default; lint sees it while it is still a fact about your source.

Expect a first pass to be mechanical, not semantic. Run binder over a real corpus and the numbers won't lie. Here are three examples, all conformant on the first pass:

  • A private 144-file engineering-docs corpus (journals, runbooks, proposals; about 115 files already carrying frontmatter, 362 relative links) converted to 144 concepts with 346 of 362 links resolved. Sixteen links stayed unresolved and were reported, two files with unparseable frontmatter were recovered as plain-markdown concepts, and fifteen concepts came out as orphans. Because those files carried their own type frontmatter, the type distribution reflected the source (mostly Note, plus a handful of proposal, reference, and subsystem). validate also surfaced real advisories: several concepts used status: active, which is not ihe OKF §5.4 vocabulary, so the bundle stayed conformant but flagged them.
  • A plugin repo of 58 markdown files produced 58 concepts and resolved 45 of 61 links, but every untyped file came out as type: Note and 31 landed as orphans. binder does not infer that a SKILL.md is a Skill; it defaults untyped files to Note and reports the orphans.
  • A small synthetic agent-exhaust folder (a TODO.md, a .plan, a memory/ folder, session notes, wikilinks, and tags) converted to 5 concepts with 9 of 11 links resolved. The unresolved pair, a [[ghost-topic]] wikilink and a broken runbooks/missing.md link, stayed in place and were reported. An authored type: Decision survived untouched, and the .plan file was skipped because binder ingests .md only.

That is binder working correctly in every case. Semantic type comes from you, through --type-map, which keys on directory, not filename:

binder convert path/to/corpus -o bundle --type-map "plugins=Skill,references=Reference"

To assist with crafting that type map, binder infer inspects the corpus and proposes candidate mappings based on a signal ladder: directory names (subsystems/ → Subsystem), filename patterns (*-spec.md → Specification), and frontmatter hints, with an optional Gemini semantic inference tier (--gemini). Because infer is strictly proposal-only, you can review its proposal, refine it, and pass it directly to conversion:

# Propose a type map from directory structure and patterns:
binder infer path/to/corpus

# Or add Gemini semantic inference:
binder infer path/to/corpus --gemini

# Apply the proposal deterministically:
binder convert path/to/corpus -o bundle --type-map "$(binder infer path/to/corpus)"

On the plugin repo, mapping plugins=Skill moved 50 files from Note to Skill. Let binder do the mechanical extraction reproducibly, but you (or your agent + skill) retain the decision on the semantics (which directory means which type, which orphan to link, whether status: active should become stable).

When you do a dry-run triage and an initial review use this as a spot check and a gate. Look for unresolved links, the orphan list, and a wall of Note-typed concepts. That'll show whether the raw exhaust made it into the bundle, and you (or your agent) decide what to designate as a type, what to link, and what to drop. Read the first review as a to-do list, not a finished bundle. binder gets you a conformant starting point in one command; turning it into a curated library is the work that follows.

One more note on messy input. binder never rejects a file. A file whose YAML frontmatter will not parse is preserved as a plain-markdown concept, stamped with a default type so the bundle stays conformant, and marked with an x_binder.recovered flag that review reads back. Agent exhaust is rarely clean; this is how binder ingests it anyway and tells you what it recovered.

Authoring OKF from scratch

When writing OKF concepts from scratch, binder is optional. If you want a tool-agnostic path without local binaries, the okf-authoring plugin (its okf-author and okf-validate skills) teaches an agent to write and check conformant OKF v0.2 concepts directly on the filesystem.

When you already have a plain-markdown tree and want deterministic frontmatter stamping and the OKF trust vocabulary, the command is enrich. It adds the required frontmatter (type, title, generated) in place, frontmatter only, no body rewriting:

binder enrich path/to/corpus --dry-run   # preview
binder enrich path/to/corpus             # write

enrich is additive and doesn't overwrite by default ("no clobber"). It'll only add keys that are absent, so an existing type or title will remain untouched, and a second run finds every key present and writes nothing. It skips a file whose frontmatter will not parse rather than rewriting it, and it writes atomically. Run it on a clean git tree and review the diff.

These three conventions help bundles align:

Use the OKF status vocabulary. OKF §5.4 defines draft, stable, and deprecated for lifecycle status. binder does not reject an unfamiliar value; it surfaces one as a validate advisory, and validate --strict turns that advisory into a CI gate. Stamp status declaratively across directory sections with --status-map, which sets status only where the field is absent:

binder convert path/to/corpus -o out --status-map "archive=deprecated,drafts=draft"

Attest with a real actor. --verified-by appends a verified stamp, and the actor has to follow the convention: human:<id>, process:<id>, team:<id>, or <producer>/<version> such as binder/0.1.0. An invalid actor is a usage error and exits 2. There is no agent: form. Say who or what verified the concept, in the vocabulary the spec understands.

Never fabricate trust. This is the invariant everything else rests on. binder derives a trust tier from the signals in the frontmatter; it never stores a credibility score. It stamps a generated: binder/<version> for content it produced, and it never auto-stamps verified. Trust mapping is off by default, so with no mapping flags the frontmatter round-trips byte for byte. When an agent drives binder, the same rule binds the agent: do not stamp trust you cannot assert.

Authoring is also where the binder Agent Skill earns its place. The deterministic CLI cannot judge which --type-map a corpus needs, or which frontmatter keys are real provenance worth mapping into sources. That judgment is what the skill carries. It complements the CLI's determinism; the mechanical, reproducible work stays in the binary.

Which surface, and why

The CLI is the foundation; the skill and the MCP server rest on the same JSON payloads it emits. Choose by the integration depth you want.

CLI. Deterministic, offline, provenance-preserving, no model in the loop. convert is SOURCE_DATE_EPOCH aware, so identical input yields byte-identical output, which is what makes hermetic golden tests and CI gates possible. Reach for the CLI for batch ingestion, pipelines, and any gate that has to run without a network or an API key. It is the deterministic core the other two surfaces build on.

Agent Skill. A skill is an authoring artifact, markdown plus scripts, that teaches an agent harness you already run (Claude Code, Cursor, Zed, or a plain script) how to drive binder for judgment-laden work: reading a dry-run triage and deciding remediate versus accept, choosing conversion flags for a corpus, reading the trust-extraction review afterward. The agent brings the semantics; binder supplies the structure and validation. Install it from binder's self-hosted marketplace: run /plugin marketplace add ghchinoy/binder in a Claude Code host and install the okf-convert plugin. It assumes the binder CLI is already on your path.

MCP server. binder mcp runs binder as a stdio MCP server, exposing its additive verbs (convert, validate, review, lint, graph) as MCP tools to a host that speaks MCP. Each tool returns the same binder.report/v1 payload as the matching binder <cmd> --json, because the handlers reuse the same internal functions and JSON encoder, so there is no second serialization path. It stays additive: source-mutating verbs (enrich, emit_concept) and read or search tools stay out, since the read surface belongs to the knowledge store and authoring over MCP is a later concern. Wire it into a host with claude mcp add binder--binder mcp, or let the okf-convert plugin's bundled .mcp.json register it on install. The skill drives the CLI from inside your harness; the MCP server hands binder's verbs to an agent runtime as tools.

Why the boundary sits where it does

That split (mechanical work in a deterministic tool, judgment with an agent) is a delegation decision, and it lines up with the tiers in DeepMind's Intelligent Delegation framework. binder's conversion is highly verifiable (the output validates against the spec) and fully reversible (it never mutates the source, and it is byte-reproducible). Work like that clears the framework's hard gates and belongs in Tier 1 or Tier 2: run it unattended, gate it in CI, poll the exit code.

The semantic choices sit a tier up. Which --type-map fits a corpus, whether a frontmatter key is real provenance, whether to remediate a broken link or accept the recovery: those are subjective and low-verifiability, so they land in Tier 3, human-in-the-loop. That is exactly what the skill encodes. The agent proposes, a human (or a --strict gate) confirms, and the irreversible-feeling decisions never happen without a checkpoint. Keeping the mechanical work in the CLI is what lets you automate it safely; keeping the judgment in the skill is what keeps a human in the loop where the framework says one belongs.

This also maps onto the choice the field guide draws out: an all-in-one agent versus bring-your-own. Google's OKF reference agent is all-in-one. It generates knowledge from a data source, runs an LLM loop, and asks only for credentials, at the cost of non-reproducible output, spend, and a network dependency. binder occupies the deterministic end. It moves knowledge that already exists into OKF, byte-reproducibly and offline, and it never invents what it cannot derive from your files. The BYO path is binder plus your own harness, driven through the CLI, the skill, or the MCP server. Pick the integration depth that fits your pipeline; the deterministic core is the same underneath all three.

Best-practices checklist

For an existing corpus, driven by the CLI:

  • Run convert --dry-run first and read the triage: concept count, unresolved links, recovered files. Decide remediate versus accept before you write the bundle.
  • Use binder infer to propose an initial --type-map from directory names and patterns, or add --gemini for semantic assistance.
  • Give the bundle a root index.md that links outward, and clear the orphans review and lint report.
  • Let cross-repo file:// links stay external; set --workspace-root only to bring genuine sibling repos inside the boundary.
  • Gate CI on --json and the exit-code contract (0/1/2/3), and add --strict when advisories should fail the build.

For authoring from scratch:

  • For a tool-agnostic path without binder, use the okf-authoring plugin (okf-author and okf-validate).
  • Use enrich when stamping required frontmatter onto an existing markdown tree; it is additive and never clobbers an authored value.
  • Keep status in the OKF §5.4 vocabulary (draft, stable, deprecated); validate --strict gates the rest.
  • Attest with a real actor (human:, process:, team:<id>, or <producer>/<version>), never an agent: form.
  • Never fabricate trust. binder derives tiers and stamps honest generated provenance; it never stores a score and never auto-stamps verified, and neither should an agent driving it.

For choosing a surface:

  • Use the CLI for deterministic, offline, CI-gateable ingestion and validation.
  • Reach for the Agent Skill when the work needs judgment and you already run an agent harness. Install it from binder's self-hosted marketplace.
  • Reach for the MCP server for interactive, persistent integration in an MCP host. Its additive convert/validate/review/lint/graph tools return the same --json payloads. Wire it with claude mcp add binder--binder mcp.

The through-line: binder keeps the mechanical, reproducible work in a tool you can audit, and leaves the semantics to you or your agent. That boundary is what makes it safe to put in a pipeline, and it holds whichever surface you reach for. It is also what turns scattered agent exhaust into knowledge another agent can use.

Further reading