Evidence Chain Specification: Every Verdict Traceable and Recomputable
The Cyclone evidence four-piece set: case hash + JSONL SHA-256 digest + failure snapshot + JUnit XML. Run the same case twice and the outputs are bitwise identical — don't take our word, verify it.
§1 · PROBLEMIn review meetings, reports lose to memories
The most common death of a test report: in a review meeting, a “pass” report loses to one sentence — “I remember it was different last week.” Without an evidence chain, a report is just paper: conclusions can’t be traced to case versions, raw data, and verdict logic; a third party can’t recompute them, and therefore can’t trust them.
§2 · SPECIFICATIONThe evidence four-piece set
Every test record is anchored by four pieces of evidence:
| Evidence | Role |
|---|---|
| case hash | The fingerprint of the case: scenario YAML + fault recipe + verdict thresholds — change one character and the hash changes |
| JSONL SHA-256 digest | Byte-level digest of the evidence log: same seed, same digest, every time |
| Failure snapshot | The signal scene at failure: last actual value, last change time, active fault-injection state |
| JUnit XML | The exit format: one <testcase> per verdict, failures carrying their snapshots, natively consumed by CI |
§3 · VERIFY IT YOURSELFPhysical proof — don't take our word, verify it
- Run the same case twice:
cyclone run cases/aeb.yaml - Compare the SHA-256 of both runs — bitwise identical, machine-checkable.
- Open
manifest.json: every verdict traces to the case version and raw data.
§4 · DEEP DIVELearn more
- The auditable test evidence chain solution page →
- How the verdict engine advances expectations through state machines →
- Why “27 out of 30” is not a pass →
§5 · HONEST BOUNDARIESHonest boundaries
- The evidence chain guarantees platform-side traceability. The SUT’s internal decision black box needs your logging cooperation before intermediate values can appear in snapshots.
- All sample figures are demo data, not measured on a real SUT.
Run it on your own scenario
A 30-minute demo: AEB regression → inject a fault → open the evidence report.
Book a 30-min demo Get the scenario pack