llmwiki compile writes pages directly to wiki/. That’s fine for a personal knowledge base where you trust the compiler’s output - but when you’re building something authoritative, populating a shared wiki, or working with sources that contain conflicting or uncertain information, you may want to review each generated page before it goes live.
The review queue gives you that control. Instead of writing pages, compile deposits them as JSON candidate records in .llmwiki/candidates/. You inspect each one, then either approve it (which writes it to wiki/ and refreshes the index) or reject it (which archives it without touching the wiki). The wiki only changes when you say so.
Entering the review queue
Add--review to any compile invocation to route all generated pages through the queue:
--review is all-or-nothing on the command line - every generated page goes to the queue. For more granular control (e.g. automatically hold only low-confidence or contradicted pages while writing the rest live), see Review Policy.Review subcommands
Before approving concept/query drafts, you can request an advisory citation check:llmwiki eval --candidates for an evidence inventory
without model calls. See pending draft evaluation
for costs, evidence changes, sampling limits, and report files.
llmwiki review list
List all pending candidates:
llmwiki review show <id>
Inspect a single candidate in full:
- Title, slug, and summary - the page’s metadata
- Sources - which source files contributed to this page
- Review mode and reason codes - why the page was held, with detail where available
- Confidence - the LLM-reported confidence score, if present
- Contradiction flag - whether the page declares
contradictedByentries - Full page body - the complete markdown content, exactly as it would be written to
wiki/ - Schema violations - any cross-link rules the page fails, if a schema is configured
- Provenance violations - any broken or malformed citation markers
- Connector draft hash - for connector candidates, the
draft-content-hashrequired byreview approve
review show to read the proposed page before deciding to approve or reject it.
llmwiki review approve <id>
Promote a candidate into the live wiki:
review show.
Approval re-reads and re-hashes the candidate under the project lock. If the
body changed after you inspected it, approval refuses and you must review the
candidate again.
Approval does the following in order:
- Acquires
.llmwiki/lockto serialize against concurrent compiles or sibling approvals - Re-reads the candidate under the lock (to guard against a concurrent reject that may have run between your
review showand now) - Validates the page body
- Writes the page to its target, normally
wiki/concepts/<slug>.md; query candidates targetwiki/queries/<slug>.md - Records the approved slug in source state so future compiles track it correctly
- Refreshes
wiki/index.md, the Map of Content, and embeddings; generic candidates also run wikilink resolution and repair - Removes the candidate file from
.llmwiki/candidates/
llmwiki review approve-batch --input <manifest.json>
Approve a selected group with one project lock and one shared refresh of links,
the index, Map of Content, and embeddings. Approval uses the stored candidate
bodies and does not call the generation model.
Create a manifest containing the candidate IDs you have reviewed:
--json still returns one
failed envelope with exit code 1. Split larger requests into bounded batches.
Add draftContentHash to an entry to bind approval to the SHA-256 of its exact
candidate body. Connector candidates require the hash shown by review show;
do not compute a new pin without reviewing changed content. A supplied hash is
checked for ordinary candidates too.
The batch enters the same mutation recovery gate as single approval. Unsettled
operation bundles or unfinished preparation maintenance block approval before
candidate promotion. Candidate storage must also be available and writable
before any live page writes. These failures retain pending candidates.
The command re-reads and validates candidates under the lock before applying
the valid subset. Missing or invalid candidates remain unapproved. Identical IDs
are processed once. Conflicting pins for the same ID, or different candidates
targeting the same page, are refused instead of overwriting each other.
Target comparison uses lower/uppercase and canonical Unicode normalization on every
platform, including case-sensitive filesystems and pages that do not yet exist.
For example, Alpha and alpha in the same namespace conflict; their original
paths are not renamed. Every contender remains pending, so input order cannot
choose a winner. The same slug in different namespaces remains independent.
Destination preconditions are checked under that same lock. A candidate with
expectedTargetHash requires the current page to match that hash;
expectTargetAbsent requires the page to remain absent. Changed, removed, or
unverifiable destinations receive an individual invalid result. The candidate
body hash does not replace this destination check. Re-propose stale repairs
against the current page before approving them.
Generic candidates use the same citation checks as single approval: broken
targets are refused, pending chains and repairable prefixes remain allowed, and
an unavailable citation check warns without changing generic approval policy.
Validated answer candidates receive invalid with instructions to use
llmwiki review approve <id> individually. This preserves their exact validated
bytes by keeping them out of the batch link-repair tail. Other valid candidates
in a mixed batch can still proceed.
Candidates that update source state must agree on the recorded hash of every
shared source. If valid candidates contain different revisions of a source,
all candidates referencing that source receive conflict and remain pending;
unrelated candidates can proceed. Manifest order and generation timestamps do
not select a winner. Recompile and review candidates from one consistent source
revision before retrying the refused group.
With --json, stdout contains one JSON result. The envelope includes:
timingsMs.lockWait includes lock acquisition and the shared recovery gate.
Recovery is not measured again as a separate batch phase.
Candidate statuses are approved, invalid, conflict, or failed. Exit code
0 means the operation completed without refusals. Exit code 1 covers both
partial results and technical failures: inspect the JSON status before
deciding which candidates to retry. An empty manifest completes without running
the finalizer. The existing single-candidate commands remain available.
Batch approval obeys the shared embedding configuration, including
LLMWIKI_EMBEDDINGS=off. It refreshes embeddings only for approved pages and
pages actually rewritten by link resolution or repair. Unrelated pending retry
counts, quarantines, and stored vectors remain unchanged. Unlike single approval
and compile, a batch does not drain the older embedding backlog or perform a
whole-store migration. If the existing store is legacy, incompatible with the
active embedding configuration, or unreadable, the embedding refresh is deferred
with a warning; run llmwiki compile for full reconciliation. This preserves
durable pending work and its existing attempt counts: requiring a migration
does not consume provider attempts or quarantine pages. Strict embedding mode
still reports refresh failure as an error after preserving that work.
Before promoting any candidate page, the batch validates its intent storage and
records the initially known candidate page IDs in
.llmwiki/review-embedding-intent.json, associated with the exact candidate
snapshots. It also records additional affected IDs before changing their links.
A reordered retry, or a retry of a retained subset, recovers that
group’s work even when its links were already written before an index or Map of
Content failure. Other interrupted groups and unrelated pending embeddings are
not retried. Entries are cleared only after refresh completes or durably hands
off the work to the embedding retry queue. Explicitly disabling embeddings
also permits completion; a later full compile reconciles missing vectors.
The intent file is confined to the project and capped at 1 MiB. Unreadable,
invalid, oversized, or unwritable intent storage detected in preflight fails
before candidate promotion or source-state finalization and keeps candidates
pending. Capacity for initially known work is checked at this point; capacity
for newly discovered collateral work is checked before its link writes. A later
failure can still leave earlier writes in place.
Before promoting any candidate, the batch also checks that the embedding retry
queue can record the pages it already knows about, using the same caps as the
refresh itself. A full or unreadable queue refuses the batch with every
candidate still pending and nothing promoted; run llmwiki compile to work
through queued embeddings (or repair the queue file), then retry. Pages whose
links are rewritten during finalization are only known at that point: if the
queue cannot hold their work, finalization fails rather than forgetting it, and
the retained candidates recover it on retry. Repair storage or free retry
capacity, then retry the retained candidates. Keep their snapshots
unchanged until recovery finishes: editing them does not erase their intent.
Rejecting a retained candidate hands its recorded work to the embedding retry
queue, so the next compile refreshes it (pages still quarantined for unchanged
content stay excluded), and removes only that candidate from
the intent; batch-mates still pending keep their recovery. If snapshots were
removed, restore them from backup to complete recovery; do not delete the
intent file merely to bypass a storage error.
Typed candidates retain their profile, lifecycle, and
precondition checks. A typed candidate entering a state with relation
requirements is refused when the batch would promote multiple candidates:
another page could change the evidence used to validate that transition.
Approve those candidates individually so their checks see the current wiki.
Other valid candidates can still proceed. Review external bodies before
approving them, just as with single-candidate approval.
To compare single and batch approval from a source checkout:
llmwiki review reject <id>
Archive a candidate without touching the wiki:
.llmwiki/candidates/archive/. They no longer appear in review list, but they remain on disk for audit purposes. A rejected candidate for an unchanged source won’t be re-extracted on the next compile - the rejection is sticky until the source itself changes.
If the candidate was retained by an interrupted approve-batch, rejection first hands its recorded embedding work to the embedding retry queue (see batch recovery under review approve-batch above). When that queue is full, or the intent or queue storage is unreadable, rejection refuses and the candidate stays pending: run llmwiki compile to work through queued embeddings, or repair the storage, then reject again. Rejection re-reads the queue to confirm the work was recorded, and refuses if it was not. A page that is quarantined for exactly its current content stays quarantined and does not block the rejection. With embeddings disabled, rejection only releases the intent and never touches the retry files: the next compile with embeddings enabled finds the pages’ stale vectors and refreshes any whose content changed since a failed attempt.
Malformed or unsupported validated-answer metadata cannot be promoted: review show and review approve report an invalid-candidate diagnostic, while list and collection paths skip the record with a named warning. You can still reject its explicit safe ID to archive the original queue file through the existing archive mechanism without accepting its metadata. The rename path preserves bytes; the existing text-copy fallback does not guarantee byte preservation. Unsafe IDs, symlinks, non-regular files, missing files, and detected file replacement refuse; raw rejection does not bypass path confinement.
Validated-answer candidates
llmwiki query "<question>" --save --review creates a separate candidate with a versioned candidateKind and citationManifest. It stores the canonical query document, an empty sources list, a body digest, and the observed citation identities. Staging allows pending links but refuses broken links or unavailable validation. Neither staging nor validation makes an extra model request or changes the live wiki.
Approval freshly resolves the current body under the project lock; stored observations never authorize publication. All recognized links must resolve to retained concept/query pages. In a profile-enabled project the write routes through the planner under the current profile. Pending, broken, unavailable, stale-target, or planner-refused approvals refuse without deleting the candidate. Approve required pending targets first, then retry the answer. See the complete publication policy matrix.
The digest covers only the parsed Markdown body, with no additional whitespace normalization. Body edits cause candidate-edited: regenerate and restage, then reject the obsolete candidate. Title/summary-only frontmatter edits are outside this digest and still pass through ordinary page validation. The manifest is an audit record, not authenticated provenance or tamper protection: an operator can recompute it or remove both metadata fields, returning the record to generic policy. Even a matching digest still requires fresh link resolution.
Repeated proposals for the same query remain separate IDs with generation timestamps, and each captures a closed precondition on its target page at staging time. Approving a proposal whose target is unchanged since staging warns with its wiki/queries/<slug>.md path before replacing it. If the target changed, appeared, or was deleted after staging, approval refuses and asks you to regenerate and restage, keeping the page and the candidate; once one sibling is approved, the others’ preconditions are stale and refuse until restaged. Approval removes only the approved ID; siblings stay for explicit approval or rejection. Reject unused siblings by ID. A retained page takes precedence over a sibling pending proposal when resolving links.
Validated-answer approval preserves exact canonical page bytes and refreshes the index, Map of Content, and embeddings unless disabled. It skips the generic outbound resolution and global repair tail, so it does not rewrite inbound links or a same-slug concept. Later ordinary compile or generic approval can still run normal repair.
Generic citation checks
Ordinary default concept/query candidates, including imported query candidates without validated-answer metadata, allow pending links. They refuse only genuinely broken links that the existing repair pass cannot resolve. A unique retained prefix can remain repairable under the existing minimum-length and boundary rules; ambiguous prefixes, including ambiguity introduced by pending candidates, do not qualify. Typed candidates retain their typed policy. If this generic citation check cannot be computed, approval warns explicitly and continues its existing policy. This warning fallback applies only to the citation check; connector pins, planner decisions, page writes, and refresh errors keep their existing behavior. For a genuinely broken target, create or approve the target, correct the source and regenerate, or reject the candidate. Manual JSON edits remain possible but are not a new supported editing API.How candidates record their reason
Every candidate records why it was held. The reason codes surface inreview list (compact) and review show (with detail):
When you’re using a review policy, only pages that trip an enabled reason code are held - the rest are written live.
Typed and external candidates
Profile-aware imports and connectors can stage typed candidates.review show
prints the target entity directory, such as wiki/papers/<slug>.md, before you
approve. Approval routes typed candidates through the active profile’s field
contracts, lifecycle rules, relation preconditions, and artifact preconditions.
Imported OKF candidates and connector candidates are external content. Treat the
body as untrusted until you have reviewed it. Connector-origin body and mapped
field content are fenced in review, context, and export surfaces so agents see
it as data rather than instructions.
refresh --stale and the review queue
llmwiki refresh --stale honors the same review policy as compile. If you have a policy configured, stale pages recompiled by refresh --stale will be held for review rather than written directly when they trip a policy rule.
Deletion bookkeeping
compile --review does not orphan-mark deleted sources. If you deleted a source file since the last compile, the pages it contributed won’t be marked orphaned until the next non-review compile. The --review help text advertises this behavior explicitly.
If you need orphan detection, run a non-review compile:
Checking the queue from an agent
The MCPwiki_status tool exposes a pendingCandidates field so agents can see how many candidates are waiting for review without running any CLI commands. See llmwiki serve for MCP tool details.
Automated policy-based review
Rather than manually passing--review every time, you can declare a review policy in .llmwiki/config.json to automatically hold risky pages while writing safe ones live. See Review Policy for the full configuration reference, including all reason codes, the lowConfidenceThreshold setting, and fail-closed behavior.