Skip to content
Benchmarks

redact-secret · Evaluation · Qualification

anthropic-api01-key

Taxonomy families: anthropic · Enterprise organization key (sk-ant-api01-). The status and its evidence, then what each scanner recorded per population.

  • View public
  • Policy revision rs-policy-1:sha256:94c247cf72ea

Support status of the product

Provisional The product’s qualification, derived by the adapter. It is not a scanner observation.

Route
No route
Evidence tier
T1
Evidence basis
provider-documented
Contract tier
T1
Fixture profile claimed
stable-documented
Fixture profile cells met
arrival-provisional, stable-documented

Why the status is not stable

  • mutation.unresolvedCritical: 2 > 0 — An unresolved critical mutation finding is a fixture-versus-detector disagreement nobody has reviewed yet; tier or sample size cannot substitute for that review.

Evidence the status was judged on

Floors and fixture-profile cells are read from public-evidence-snapshot alone; a gate reads each gate-bearing population on its own.
Scored cases
37
Positive cases
15
Positive context axes
15
Benign controls
15
Benign control axes
6
Twin pairs
7
Metamorphic critical failures
0
Mutation unresolved critical
2
Differential unresolved disagreements
0

Zero-tolerance gates

The classifier receives the worst gate-bearing population, never a sum. Each population is shown on its own.
Zero-tolerance gates
PopulationTwin pairs that did not discriminateBenign controls flagged
public-evidence-snapshot0 of 70 of 15
regression-corpus0 of 2No control

Scanner observations

What each scanner recorded for this family, per population. These are counts and carry no support status; a case that is pending or not measured is not a miss.
Scanner observations
PopulationScannerCasesPositive spansSpan outcomesLeakedBenign controls flaggedTwin pairs discriminatedPending / not measured
policy-corpuspolicy routeflare-redact0No caseNo case in this populationNo positive spanNo controlNo pairNone
policy-corpuspolicy routegitleaks0No caseNo case in this populationNo positive spanNo controlNo pairNone
policy-corpuspolicy routeredact-secret0No caseNo case in this populationNo positive spanNo controlNo pairNone
policy-corpuspolicy routetrufflehog0No caseNo case in this populationNo positive spanNo controlNo pairNone
public-evidence-snapshotfloors and gatesflare-redact3715EXACT 150 spans · 0 bytes0 of 156 of 7None
public-evidence-snapshotfloors and gatesgitleaks3715EXACT 2 · MISS 1313 spans · 1,308 bytes1 of 150 of 7None
public-evidence-snapshotfloors and gatesredact-secret3715EXACT 150 spans · 0 bytes0 of 157 of 7None
public-evidence-snapshotfloors and gatestrufflehog3715MISS 1515 spans · 1,524 bytes0 of 150 of 7None
regression-corpusgatesflare-redact42EXACT 20 spans · 0 bytesNo control2 of 2None
regression-corpusgatesgitleaks42MISS 22 spans · 216 bytesNo control0 of 2None
regression-corpusgatesredact-secret42EXACT 20 spans · 0 bytesNo control2 of 2None
regression-corpusgatestrufflehog42MISS 22 spans · 216 bytesNo control0 of 2None

Cases

Each case of this family, per population, with what every scanner recorded for it and the case’s own facts.

Open the cases of anthropic-api01-key