Skip to content
Benchmarks

redact-secret · Evaluation · Qualification

generic-token

Taxonomy families: No provider · Unclassified assignment literal. The status and its evidence, then what each scanner recorded per population.

  • View public
  • Policy revision rs-policy-1:sha256:94c247cf72ea

Support status of the product

Provisional The product’s qualification, derived by the adapter. It is not a scanner observation.

Route
No route
Evidence tier
T3
Evidence basis
project-policy
Contract tier
T3
Fixture profile claimed
Not recorded
Fixture profile cells met
arrival-provisional, stable-documented, stable-empirical, context-constrained-empirical

Why the status is not stable

  • policy.positive-cases: 2 (requires 6)
  • policy.positive-axes: 2 (requires 4)
  • policy.benign-cases: 1 (requires 8)
  • policy.benign-axes: 1 (requires 4)
  • policy.twin-pairs: 1 (requires 5)
  • policy.critical-failure: 294 (requires 0)
  • policy.protected-holdout: not-run (requires pass on frozen candidate)
  • benign.falseAlarms: 1 > 0 — A false alarm on a benign control is a recorded false positive; stable tolerates none.
  • metamorphic.criticalFailures: 252 > 0 — A metamorphic invariant violation (for example, detection lost after an envelope-preserving transform) shows the detector is not robust to construction, independent of how many cases were run.
  • mutation.unresolvedCritical: 36 > 0 — An unresolved critical mutation finding is a fixture-versus-detector disagreement nobody has reviewed yet; tier or sample size cannot substitute for that review.
  • differential.unresolvedContractDisagreements: 6 > 0 — An unresolved disagreement with a peer scanner over what the provider contract requires means the contract itself may be wrong; stable requires it settled.

Evidence the status was judged on

Floors and fixture-profile cells are read from public-evidence-snapshot alone; a gate reads each gate-bearing population on its own.
Scored cases
221
Positive cases
115
Positive context axes
49
Benign controls
93
Benign control axes
16
Twin pairs
13
Metamorphic critical failures
252
Mutation unresolved critical
36
Differential unresolved disagreements
6

Zero-tolerance gates

The classifier receives the worst gate-bearing population, never a sum. Each population is shown on its own.
Zero-tolerance gates
PopulationTwin pairs that did not discriminateBenign controls flagged
public-evidence-snapshot0 of 131 of 93
regression-corpusNo pair0 of 44

Scanner observations

What each scanner recorded for this family, per population. These are counts and carry no support status; a case that is pending or not measured is not a miss.
Scanner observations
PopulationScannerCasesPositive spansSpan outcomesLeakedBenign controls flaggedTwin pairs discriminatedPending / not measured
policy-corpuspolicy routeflare-redact42MISS 22 spans · 40 bytes0 of 10 of 1None
policy-corpuspolicy routegitleaks42EXACT 1 · MISS 11 spans · 12 bytes0 of 11 of 1None
policy-corpuspolicy routeredact-secret42EXACT 20 spans · 0 bytes0 of 11 of 1None
policy-corpuspolicy routetrufflehog42MISS 22 spans · 40 bytes0 of 10 of 1None
public-evidence-snapshotfloors and gatesflare-redact237115EXACT 1 · MISS 114114 spans · 5,765 bytes1 of 933 of 1316 pending
public-evidence-snapshotfloors and gatesgitleaks237115EXACT 53 · PARTIAL 1 · MISS 6162 spans · 3,781 bytes0 of 9313 of 1316 pending
public-evidence-snapshotfloors and gatesredact-secret237115EXACT 80 · MISS 3535 spans · 2,834 bytes1 of 9313 of 1316 pending
public-evidence-snapshotfloors and gatestrufflehog237115MISS 115115 spans · 5,805 bytes0 of 930 of 1316 pending
regression-corpusgatesflare-redact7026EXACT 3 · PARTIAL 1 · MISS 2223 spans · 761 bytes1 of 44No pairNone
regression-corpusgatesgitleaks7026EXACT 10 · PARTIAL 1 · MISS 1516 spans · 533 bytes0 of 44No pairNone
regression-corpusgatesredact-secret7026EXACT 260 spans · 0 bytes0 of 44No pairNone
regression-corpusgatestrufflehog7026EXACT 2 · PARTIAL 1 · MISS 2324 spans · 801 bytes0 of 44No pairNone

Cases

Each case of this family, per population, with what every scanner recorded for it and the case’s own facts.

Open the cases of generic-token