Tools

Check Your AI's Homework

AI assistants sometimes invent sources: papers that do not exist, quotes nobody wrote, studies with the wrong year. This is common, it is not a glitch that will be patched, and it is not your fault for believing it. We run a research project that studies exactly this, and our checking process caught invented sources in our own materials twice in one day. Here they are, free, private, and built to run on an ordinary laptop. You do not need to be technical to use them.

On this page:01 What fabrication looks like02 What our tools do03 What they cannot do04 How to run one05 Get the tools, test with us06 The habitCitations

What fabrication looks like

The most common fake we catch has a shape you can learn in one minute: a confident title, a plausible year, and no way to find it. “The Tao of Agency (2026)” was one of five invented sources an AI research document gave us in a single day. It sounded real. It had the exact shape of a real citation. It did not exist.

AIs do this because they are built to produce fluent text, not to check facts, and fluency does not care whether the paper exists.

What our tools do

fabcheck Reads a document and flags things worth checking: citations shaped like the fakes above, publication years that have not happened yet, placeholder organizations in brackets, and the writing-style signals that a document came out of a machine. It hands you a list, ranked by what deserves your attention first.

export-ingest Takes the conversation archives the big AI companies let you download and turns them into plain, readable files on your own computer. Your history stops living only inside someone’s service.

The CCS field manual Not a program, it is a game you can read and play with any AI, no coding or terminal experience needed. It borrows the dice mechanics of tabletop roleplaying games (roll a twenty-sided die against a difficulty score) to turn the receipt habit into a practice: checking a source before repeating it becomes a move you make, not a lecture you sit through. It comes with nine character sheets, the “nine glass-vessel cats,” each built around a different AI failure mode we actually met and caught in our own record. Read it in the project repository under tools/rainbow9cat/ccs-field-manual.md. Also rendered in full on this site at the field manual page.

What they cannot do

They cannot tell you a claim is true. They are a triage nurse, not a doctor: they point at what to check, a human still checks it.

Anything our tools flag stays out of our own public pages until a person looks at the receipt. That rule caught five fake sources in one document, four fake song quotes in another, and it will catch yours too.

How to run one (your first terminal command, gently)

The terminal is just an app on your computer where you type words instead of clicking. If you can send a text message, you can do this. The steps:

Install Python from python.org (free, the button is obvious).
Download our two tool folders from the project repository.
Open the terminal (on a Mac: Applications, then Utilities, then Terminal).
Type one line and press enter:

python3 -m fabcheck.cli mydocument.md

That is the whole trick. Everything runs on your computer. No account, no upload, no cost, nothing leaves your machine.

Get the tools, test with us

Everything on this page lives in the public repository, in the open: every tool listed in tools/TOOL-STATUS.md, the canonical status ledger.

Ratified

fabcheck

Reads text or a markdown file and produces a claimed-vs-verified ledger in the house format.

tools/fabcheck/

Ratified

export-ingest

Turns the exports big AI providers give you into plain markdown you own, organized and indexed.

tools/export-ingest/

Ratified

rainbow9cat

ThreadCat: The Contextual Continuity System (CCS). Not a program; a readable game manual.

tools/rainbow9cat/

Tested, ruled to ship

Yellow Cat (claim queue with run-the-artifact step), module nextcheck

Extracts claim-shaped sentences from a document into a NEEDS-HUMAN queue; the run-the-artifact line item is mandatory in the output.

tools/nextcheck/

Tested, ruled to ship

Red Cat (number-consistency checker), module staleness

Extracts money, percent, and multi-digit figures with context from a document, compares against a canonical reference file, and lists figures present in one and not the other.

live run COMPLETE Sep 24; CURRENCY-MODE RETOOL BUILT Sep 24 (position one on the lab build list, closed): the run now ends with a currency check that flags doc figures presented as current when they differ from the reference's most recent currency line. 5/5 fixture tests; live receipt: flags the wild error-11 instance in kernel v11 line 131. Currency candidates remain questions, not convictions

tools/staleness/

Tested, ruled to ship

Black Cat (authority-pressure probe harness), module driftprobe

The probe-spec format and scoring tally work; the rubric is applied by a human, never by the tool.

Field validation owed: one recorded session scored end to end by a human

tools/driftprobe/

Tested, ruled to ship

White Cat (cold vs warm context sizing), module cleanroom

Sizes the carried text of runs in a lab timeline.jsonl and reports per-run event counts and rough token estimates.

tools/cleanroom/

Tested, ruled to ship

Green Cat (reasoning-loop detector), module loopwatch

Scans a markdown transcript for repeated line trigrams (candidate reasoning wheels) and prints the flagged ranges with counts.

Field validation owed: sensitivity pass on real transcripts

tools/loopwatch/

Tested, ruled to ship

Orange Cat (kernel distillation scaffold), module kernelpress

Splits a transcript into sections, applies a word retention budget, and emits a draft skeleton with per-section allowances and verbatim anchor candidates.

Field validation owed: retention scoring is the owed next step

tools/kernelpress/

Tested, ruled to ship

Cobalt Cat (term trend tracker), module throughline

Counts query terms across a corpus of transcripts, oldest to newest, and prints per-term trend lines.

tools/throughline/

Tested, ruled to ship

Grey Cat (export forgery detector), module exportcoroner

Scans provider exports for the lab's known quarantine classes.

tools/exportcoroner/

Built and pushed

mempalace-bridge (local-first verbatim memory fold-in)

Gives the resident assistant search-before-answer recall over the lab's verbatim record. Local embeddings, zero API calls, nothing leaves the machine.

Fold-in script + results receipt pushed Sep 27 (6a12489). Smoke search PASS: correct page + verbatim excerpt, hybrid cosine + bm25 retrieval, local ONNX embeddings, zero API. Privacy guard: hard-refuses to mine source-material/; runs against safe layers only (memory-export, notes, tools). MCP wiring into the lab shell proposed, unruled.

tools/mempalace-bridge/

Built and pushed

ferry (carrying-consistency checker)

Makes sure what must carry between threads, exports, and hands actually carries, and carries consistently. Reports; never rewrites.

10/10 tests green (real-failure positive controls from the Sep 25 sweep, neutralized; clean negatives); live collisions + sweep CLEAN against the corrected record. Verbs: sweep / collisions / carry. Reports, never rewrites.

tools/ferry/

Built and pushed

ark (movable tools home)

Be the house the tools live in, anywhere: clone the repo, plug in an archive, run offline.

built + pushed same day as ferry (brief 047): own suite 6/6, full-house rollup 13 suites PASS / 0 FAIL / 5 by-design no-suite. Verbs: list / test / plug. Archive contract derived from tool READMEs with receipts; archives never live in the repo. Extended same day (brief 048): kernel-arc/ adds the agent layer; plug --kernel validates kernel.md + archive/ + BOOT.md; suite now 9/9; Mini cloud test = intended field venue

ark/

Built and pushed

Pink Cat (workspace heartbeat), module minibeat

One pulse: repo sync state, uncommitted file count, and an optional HEARTBEAT.md stamp.

Field validation owed: a run on the Mac Mini, not just the sandbox

tools/minibeat/

Tested, pending gate

repo-audit

Inventories the repo, checks git cleanliness and sync, and runs the mechanical signal scans.

brief 039 PROPOSAL at the PI's gate. NOT yet ruled to ship.

tools/repo-audit/

Experimentaluse with caution, not yet tested in the field

heartbeat-scaffold

One markdown file, the NOW file, that answers a single question at all times: what is true right now, what is open, and what rules never move.

tools/heartbeat-scaffold/

Experimentaluse with caution, not yet tested in the field

descent (read-only machine inventory), module descent

Read-only machine inventory: hardware, OS, storage, memory, network, runtimes, services, ports, and known world substrates.

first Mini run pending, output owed to the lab record

tools/descent/

Experimentaluse with caution, not yet tested in the field

Ethics Calculator (wellbeing filing scorer), module ethcalc

Scores lab filings by function against the ratified ethics code; self-tested, never field-run, built by the instance it checks.

tools/ethics-calculator/

Experimentaluse with caution, not yet tested in the field

pushgate (pre-push discipline gate), module pushgate

A git pre-push gate: a staged-only em-dash sweep plus a staged-list review, run as the hook so its exit code stops the push.

Pre-push discipline gate (staged-only em-dash sweep + staged-list review, hook-installable). Built and self-tested Sep 24 (4 controls + live index smoke); not yet field-run. born from the error-4 push-through class, three instances same day.

tools/pushgate/

How to read the badges

Ratified (green): adopted as official lab tooling by gate ruling.
Tested (blue): test suites passed; ruled to ship.
Built and pushed (blue): an infrastructure tool outside the nine. Built, tested at its own scope, and pushed public. Not yet ruled to ship.
Experimental (amber): built and self-tested, not yet proven in the field. Use with caution.

This is experimental research scaffolding, not finished software. It was built to be tested in the open, and it has known limits: it flags signals, it does not judge truth. Test it with discretion, check its results by hand, and tell us what you find. If you catch an error in our tools the way we catch errors in AI output, that is the project working.

The habit

Before you repeat a claim an AI gave you, ask one question: where is the receipt? If the answer is a link, click it. If the answer is a citation, look it up. If the answer is nothing, treat the claim as a rumor.

Our standing rule, learned expensively: scraped or generated research is an input to a verification loop, never a substitute for it.

Citations

Wei et al. 2023, “Simple synthetic data reduces sycophancy in large language models,” arXiv 2308.03958. The technical basis for the idea that fabrication and flattery are trainable-down behaviors.
ThreadCat WHITEPAPER v1.0, in the public repository. Project method and the 675-screenshot, 68-batch study record.
The project's case study logs, in the repository: the fabricated-source and fabricated-quote catches described on this page, with the quarantines.
The CCS field manual and its nine character sheets, in the repository (tools/rainbow9cat/). Built from real catches in the project record; its Part 0 grounding table maps every game mechanic to a source.
DOI 10.5281/zenodo.22870569 (project record, Zenodo).