Download & Cite

The benchmark artifact is the frozen evaluation-v1.0 suite: 350 scenarios with per-prefix normative references, the manifest that pins their hashes, and the normative specification they were generated from. All three files below are byte-identical copies of the frozen release; the SHA-256 lets you confirm that.

Files

Scenario suite (JSONL)
Download

One JSON record per line. Each record holds the agents, objects, observations and relations of a scenario plus the full evaluation block: reference posterior, categorical state, probability direction, evidence attribution, contradiction state and reference action at every evidence prefix.

6.7 MiB (7,069,864 bytes) 350 records
sha256 8614d33048b6885c391c9ffe3bd152a297de1fe885e8bbb12b355c538080225c
Benchmark manifest
Download

Scenario IDs, per-scenario SHA-256 hashes, semantic fingerprints, prompt-template hashes, the model panel, decoding settings, the seven freeze gates and the post-freeze correction policy.

117 KiB (119,463 bytes)
sha256 8f01af84ef1d2ce298d26789aed0c1882f87e84d77167dbe47fd1bbb65c7c536
Normative specification
Download

The checked-in rules every reference label is derived from: state space, prior, likelihood semantics, temporal and supersession rules, relation transfer, contradiction rules, action rules and per-regime construction constraints.

4 KiB (4,248 bytes)
sha256 c1917910281e3fdbf5d6432181e68e3724dd5916e8e290f2f435d6ff11198b32
Evidence-Graph-Toolkit @ evaluation-v1.0
Open

The immutable git tag with the generator, the independent log-odds checker, prompt templates, renderers for all nine representation conditions, metrics, statistics and every figure and table script.

Verify a download on the command line:

Terminal window
shasum -a 256 creg-evaluation-v1.0-scenarios.jsonl
# expected: 8614d33048b6885c391c9ffe3bd152a297de1fe885e8bbb12b355c538080225c

Load it in Python:

import json


with open("creg-evaluation-v1.0-scenarios.jsonl") as f:
    scenarios = [json.loads(line) for line in f]


s = scenarios[0]
print(s["id"], s["metadata"]["trajectory_regime"])
for step in s["evaluation"]["belief_trajectory"]:
    ref = step["reference_beliefs"][0]
    print(step["step_index"], step["new_evidence_ids"], ref["reference_distribution"])

Regenerate the suite

The suite is deterministic. Regenerating it from the tagged toolkit must reproduce every record hash in the manifest; the same command also re-runs the independent checker that reconstructs every prefix posterior in log-odds space.

Terminal window
git clone https://github.com/RanulfoBezerra/Evidence-Graph-Toolkit
cd Evidence-Graph-Toolkit
git checkout evaluation-v1.0
python -m pip install -e .
python experiments/prepare_evaluation_v1.py \
  --scenario-seed 20260725 \
  --scenarios-per-regime 50 \
  --output-dir results/evaluation_v1_preparation

The output directory contains scenarios.jsonl, the checker report and a draft manifest whose scenario_hashes should match the frozen manifest above.

Cite

BibTeX 
@misc{bezerra2026creg,
  author       = {Bezerra, Ranulfo and Yamazaki, Kimitoshi},
  title        = {A Benchmark and Layered Evaluation Protocol for Evidence-Grounded
                  Shared Belief Updating in Connected Robotics},
  year         = {2026},
  howpublished = {\url{https://github.com/RanulfoBezerra/Evidence-Graph-Toolkit}},
  note         = {Connected Robotics Evidence Graph (CREG) Benchmark, frozen as evaluation-v1.0}
}

The manuscript is under review. The paper sources live in the toolkit repository under paper/; the entry above will be updated with the venue once it is public.

License and terms

The toolkit, the scenario suite, the references and the prompts are released under the MIT License (© 2026 Ranulfo Bezerra). The suite is synthetic and contains no personal data. The benchmark is frozen: post-freeze corrections require a changelog entry and a protocol-version increment, and never silently change a score. If you find an issue, please open an issue rather than patching labels locally.