Test-Type Matrix: One Engine, Four Test Types

Functional, performance, reliability, and environmental testing share one scenario YAML, scheduler, verdict engine, and evidence chain — what changes is the clock mode, the metric set, and the fault recipe. One report format, one CI gate.

§1 · PROBLEMFour teams, four report formats

In the traditional setup, functional testing is the scripting team’s scripts, performance is the fixture team’s fixtures, reliability is the rig team’s long runs, and environmental testing is the proving ground’s chambers — four teams, four report formats, four different meanings of “pass”. At DV/PV roll-up, nobody can assemble a single evidence chain.

§2 · MATRIXFour test types, one foundation

All four share the same scenario YAML, scheduler, verdict engine, and evidence chain — what changes is the clock mode, the metric set, and the fault recipe:

• §3.1 Functional Supported: offline mode on a virtual clock — rerunning the same case is bitwise identical. Statistical metric verdicts are supported; per-event assertions (“brake must fire before TTC < 1.2 s”) are planned.
• §3.2 Performance Prototype: online mode on the real clock, hardware timestamps at bus ingress/egress. Boundary: CPU load and WCET belong to OS-level profilers — Cyclone owns bus-visible end-to-end latency.
• §3.3 Reliability Supported: a seed matrix of 300 runs feeding Wilson/SPRT — only 300/300 supports a ≥ 99% reliability claim at 95% confidence; soak runs watch memory leaks and counter overflows.
• §3.4 Environmental Prototype: heat, vibration, EMC, salt spray (ISO 16750 / GB/T 28046) belong to climate chambers and rigs; but every environmental stress eventually projects onto the bus, and that projection is the software half.

§3.5 Env-stress projection

While the sample bakes in the chamber, the fault campaign runs on the rig, and both evidence sets bind into one DV/PV report.

§4 · WHY IT MATTERSThe compounding of one foundation

All four reports share one format, one hash-verifiable chain, and one CI gate — adding a test type means writing new YAML, not new tooling.

• How scenarios become CI regression: ADAS scenario regression testing →
• How environmental stress becomes fault recipes: deterministic fault injection →
• The statistics behind reliability verdicts: Wilson lower bound →

§5 · HONEST BOUNDARIESHonest boundaries

• Physical environmental testing is not something software can do — we only do its projection onto the bus.
• Performance testing is a prototype: hardware timestamping depends on the specific bus card; we evaluate against your hardware during a POC.
• Thresholds need calibration: the figures here are demo values; production acceptance thresholds are derived by margin analysis against your SUT baseline.