Section pages
Evidence
Trust should come with evidence.
Research and architecture in development. This page explains our design direction. It is not a list of completed PRAXIS capabilities. See the documented product capabilities and open work.
Useful context must survive checking and reuse. Our evaluation work examines extraction, citations, time, portability and review effort. This page distinguishes published artifacts, recorded PRAXIS operations and analytical quality we have yet to establish.
Benchmark methodology published June 2026 · status copy revised September 30, 2026
- publicCorpus and task definitions — 14 task types in v0.1, in the open repository.
- publicMethodology and scoring design.
- pendingFrontier-model baseline runs — first scores to appear on this page.
- pendingTACITUS submission — waits on the capsule engine’s eval harness.
Does the next brief cost less effort?
Compare equivalent sources across a first and second assignment, with a second analyst and an independent domain reviewer. Record citation entailment, missing caveats, actor/date/modality errors, source-finding time, representation switching, elicitation, setup, licensing, maintenance and reviewer time. Check impact false positives and missed dependencies. Count all preparation and upkeep before calculating net effort.
| Outcome | Measured result |
|---|---|
| Source/citation entailment | |
| Caveat and actor/date/modality errors | |
| Setup + review + maintenance effort | |
| Second-analyst repeat usage | |
| Impact false positives / missed dependencies | |
| Net effort |
The hypothesis weakens if preparation outweighs repeat savings, reviewer burden persists, imported methods create unsupported confidence, caveats disappear or analysts abandon reuse. A synthetic replay or a successful runtime operation does not establish a customer result.
Different evidence answers different questions
- Authored illustration: teaches a design idea. No model or deployed workflow is established.
- Compiled reference fixture: shows a concrete artifact and format checks. It does not prove product integration or factual accuracy.
- Real-service acceptance: tests a workflow against a dated release with synthetic private inputs. It does not establish representative analytical quality.
- Measured comparison: needs the same source set, named baselines, a scoring rubric and reproducible runs. No new comparative scores are published by this redesign.
- Customer or independent user study: requires permission, real tasks and attribution. Fictional cases are not customer deployments.
PRAXIS acceptance recorded September 30, 2026
The source record for release 4480650999b5bd2808b55cba58f16e5a9a83d343 reports existing-account login; TXT/PDF/DOCX upload and capsule review; separate private save-to-Ask; conversation and working-note reuse; queued research and owner approval; and memo edit/save/reopen. Saved context and history were reloaded. Screens from those checks are available in the product tour.
Generated drafts remained review-required. Review found funding-versus-expenditure confusion, unsupported deductions and inaccurate institutional/source attribution. Corrections were saved, but requested original-source substantiation was incomplete. The operational pass does not certify analytical correctness.
Not established by those checks: fresh signup/reset, expiry/deletion, two-account switching, purchases and paid entitlements, inventory capacity, scanned OCR, or interrupted-worker recovery. Fresh privileged cloud-log inspection also awaited restored operator authentication.
This public summary omits private account and artifact identifiers. It summarizes the recorded acceptance evidence; it is not a new live canary. Download the dated, versioned summary.
Analytical-quality comparison protocol · Visitor-comprehension protocol. Protocols are proposed work, not results.
What gets measured
Typed extraction F1
Did the system produce the right primitives with the right classes — actors, claims, commitments, events — not just fluent prose about them?
Edge fidelity
Are the relations typed and correct? “Rejects”, “commits-to”, and “contradicts” are different edges with different consequences.
Span fidelity
Does each cited span actually contain the claim attributed to it? Catches the failure citation features hide: confident text pointing at the wrong receipt.
Temporal correctness
Both clocks present and right; interval relations between episodes consistent; superseded claims chained, not erased.
Contradiction preservation
Conflicting claims must survive as an explicit typed edge. Smoothing a dispute into one confident sentence scores zero on the item.
Why these five
These checks examine whether a structured answer preserves the source record. They do not establish overall analytical quality, real-world impact, or performance outside the evaluated cases.
Reproduce it yourself
git clone https://github.com/sargonxg/TCGC_TACITUS-Conflict-Grammar-Corpus_BENCHMARK tcgc cd tcgc # corpus items: items/v0.1-sample/ # task schema: schema/tcgc-v0.1.json # run your system over an item, emit the typed subgraph, score per the schema
If a published score on this page cannot be reproduced from the repository, treat it as wrong and tell us: hello@tacitus.me.
No placeholder numbers
A number appears on this page only when it comes from a published, reproducible run. Until then, the state is “in progress” and says so.
Compare against clear baselines
The first published scores will be frontier-model baselines on the open corpus — including where they do well. Our own submission lands when the eval harness is complete.
Publish enough to reproduce a result
Corpus, task definitions, and scoring all live in the open repository. If you can’t re-run it, we don’t cite it.