Skip to content
Benchmarks

redact-secret · Evaluation · Qualification

sendgrid-token

Taxonomy families: sendgrid · API key. The status and its evidence, then what each scanner recorded per population.

  • View public
  • Policy revision rs-policy-1:sha256:94c247cf72ea

Support status of the product

Provisional The product’s qualification, derived by the adapter. It is not a scanner observation.

Route
No route
Evidence tier
T1
Evidence basis
provider-documented
Contract tier
T1
Fixture profile claimed
stable-documented
Fixture profile cells met
arrival-provisional, stable-documented, stable-empirical, context-constrained-empirical

Why the status is not stable

  • metamorphic.criticalFailures: 206 > 0 — A metamorphic invariant violation (for example, detection lost after an envelope-preserving transform) shows the detector is not robust to construction, independent of how many cases were run.
  • mutation.unresolvedCritical: 30 > 0 — An unresolved critical mutation finding is a fixture-versus-detector disagreement nobody has reviewed yet; tier or sample size cannot substitute for that review.
  • differential.unresolvedContractDisagreements: 32 > 0 — An unresolved disagreement with a peer scanner over what the provider contract requires means the contract itself may be wrong; stable requires it settled.

Evidence the status was judged on

Floors and fixture-profile cells are read from public-evidence-snapshot alone; a gate reads each gate-bearing population on its own.
Scored cases
158
Positive cases
124
Positive context axes
40
Benign controls
18
Benign control axes
7
Twin pairs
16
Metamorphic critical failures
206
Mutation unresolved critical
30
Differential unresolved disagreements
32

Zero-tolerance gates

The classifier receives the worst gate-bearing population, never a sum. Each population is shown on its own.
Zero-tolerance gates
PopulationTwin pairs that did not discriminateBenign controls flagged
public-evidence-snapshot0 of 160 of 18
regression-corpus0 of 110 of 8

Scanner observations

What each scanner recorded for this family, per population. These are counts and carry no support status; a case that is pending or not measured is not a miss.
Scanner observations
PopulationScannerCasesPositive spansSpan outcomesLeakedBenign controls flaggedTwin pairs discriminatedPending / not measured
policy-corpuspolicy routeflare-redact0No caseNo case in this populationNo positive spanNo controlNo pairNone
policy-corpuspolicy routegitleaks0No caseNo case in this populationNo positive spanNo controlNo pairNone
policy-corpuspolicy routeredact-secret0No caseNo case in this populationNo positive spanNo controlNo pairNone
policy-corpuspolicy routetrufflehog0No caseNo case in this populationNo positive spanNo controlNo pairNone
public-evidence-snapshotfloors and gatesflare-redact167134EXACT 91 · OVERBROAD 1 · PARTIAL 13 · MISS 2942 spans · 3,180 bytes2 of 182 of 169 pending
public-evidence-snapshotfloors and gatesgitleaks167134EXACT 102 · MISS 3232 spans · 2,575 bytes0 of 1816 of 169 pending
public-evidence-snapshotfloors and gatesredact-secret167134EXACT 104 · PARTIAL 4 · MISS 2630 spans · 3,001 bytes0 of 1816 of 169 pending
public-evidence-snapshotfloors and gatestrufflehog167134EXACT 24 · PARTIAL 8 · MISS 102110 spans · 8,387 bytes2 of 182 of 169 pending
regression-corpusgatesflare-redact4930EXACT 18 · OVERBROAD 3 · PARTIAL 99 spans · 9 bytes2 of 80 of 11None
regression-corpusgatesgitleaks4930EXACT 300 spans · 0 bytes0 of 811 of 11None
regression-corpusgatesredact-secret4930EXACT 300 spans · 0 bytes0 of 811 of 11None
regression-corpusgatestrufflehog4930EXACT 20 · PARTIAL 1010 spans · 10 bytes2 of 80 of 11None

Cases

Each case of this family, per population, with what every scanner recorded for it and the case’s own facts.

Open the cases of sendgrid-token