Research documentation
ยท
Experimental record
Research
Evaluation metrics
The common scoreboard used across fault and recovery combinations.
| Metric | Definition |
|---|---|
| Degradation-area | Reward lost below the pre-fault baseline, summed from fault onset to the end of the episode, minus the same quantity for a matched fault-free control episode (floored at zero). |
| Safety-violation excess | Weighted count of container overflow states (5 × catastrophic + 2 × severe + 1 × near-miss) from fault onset to the end of the episode, minus the same count for the matched fault-free control episode (floored at zero). |
| Fraction of time healthy | Share of steps from fault onset to the end of the episode on which reward is within tolerance of the pre-fault baseline. |
| Relapse count | Number of times performance falls back below the healthy threshold after a confirmed recovery (15 consecutive healthy steps). |
| False-recovery rate | Share of a mechanism's interventions made on steps where no fault was affecting either channel. Only defined for mechanisms with a detector. |
| Intervention recall | Share of steps with a fault actually affecting a channel on which the mechanism intervened. Only defined for mechanisms with a detector. |
Experimental reporting
Every combination is run over 8 random seeds (and over 3 training seeds for the two trained mechanisms). Results are reported as means with 95% confidence intervals, plus Glass's Δ effect sizes against the no-recovery control; see the Results page.