LLM Council

What I built

I developed a macOS command-line workflow that collects independent answers, organizes blind peer review, and records how a final synthesis uses or rejects each contribution. The resulting report keeps disagreement and uncertainty visible alongside the recommendation.

I wanted more than a fluent final answer: a reader should be able to inspect the alternatives, the reviews, and the decision process. The orchestration therefore treats structured reviews, incomplete runs, and local records as part of the product.

Explore the fictional example

The browser walkthrough presents a fixed community-garden fixture with authored mock responses. Three council members consider a $900 donation, review one another, and produce a recommendation that retains a dissenting view.

These responses are not a recorded live model run, and the page sends no prompts or paid requests. The repository also provides an offline demo that exercises the real orchestration contract with mock adapters.

Architecture and decisions

• Separate preview from execution. The CLI previews planned work and cost and requires explicit confirmation before execution; rejected reviews are not retried automatically.
• Make reviews accountable. Neutral aliases hide authorship from reviewers, self-review is excluded, and local validation checks structured review contracts.
• Expose incomplete work. Results distinguish full-roster, quorum-degraded, and partial completion; recorded cost can be a lower bound after a failed invocation.
• Preserve disagreement. Reports identify adopted contributions, rejected recommendations, remaining dissent, and uncertainty.
• Keep local records deliberate. Keychain stores credentials, while SQLite and Markdown/JSON exports retain inspectable run history with explicit retention behavior.

My contribution

I defined the use case, deliberation requirements, scope, and acceptance decisions, using the AI-assisted implementation and review approach described on the About page.

Scope and verification

The September 22 release record reports 60 passing offline tests; this is recorded evidence rather than a fresh run by the website, and the documented live acceptance result remains quorum-degraded. The browser fixture illustrates review structure without evaluating model quality or current provider access. Multiple models are not necessarily independent in training, unbiased, or correct, and these results do not establish that a council outperforms a single model.

Run the offline example

From a checkout with Python 3.14:

python3 -m venv .venv
.venv/bin/pip install .
.venv/bin/python examples/offline_demo.py --output-dir demo-output

The command writes mock JSON and Markdown run exports; real provider execution belongs to the standalone CLI with separate credentials and confirmation.

Source snapshot ea9cfa4 · Reviewed 2026-09-22. This case study describes that revision; development may continue separately.