EVALUATION METHOD

Holdout

Frozen cases run in isolation against a frozen candidate. Only aggregate counts cross the publication boundary, so there is no case to open. Execution-qualified means the infrastructure ran its contract, not that a scanner detects well.

Qualification run 19510b39 · 2026-09-29
Source Frozen report in the repository

1 · How it runs

Did the frozen candidate run its holdout cases?

InputFrozen public control cases, sealed at execution.
ChangeNone.
CheckEach scanner is read against the cases once. Counts are kept per stratum: the expected kind and tier.

Cases: 12
Variants: 12
Scanners: 3

2 · Recorded now

What the holdout run recorded

Execution qualified (engine-conformance), run 19510b39 on 2026-09-29. It records that the infrastructure executed its contract. It makes no detection-quality or support claim.

Holdout checks per scanner and stratum
Stratum	redact-secret0.1.0-beta.11	gitleaks8.30.1	trufflehog3.97.4
Must not flag, tier T3	0 of 6	0 of 6	0 of 6
Must redact, tier T1	0 of 6	0 of 6	0 of 6

Checks that did not hold, of those scored for that scanner.

3 · How to read it

Reading holdout

This page reads a qualification run, which is a separate snapshot from the discovery run the other methods read.
Public controls cannot establish independent detector performance, and the aggregate has no detector attribution or unique affected cases.

4 · Exact inputs

The corpus and the candidate

A holdout case cannot be opened. These are the aggregate facts the report publishes.

Corpus: engine-v1-public-controls, revision 1
Purpose: public-conformance
Lifecycle: sealed-at-execution
Independence: public-control
Methodology: frozen-candidate-canonical-cases-aggregate-only
Cases: 12 cases, 12 variants, 0 generation errors
Plan: 11390c307f4d

Full hashes of the corpus, the plan and the candidate

{
  "corpusHash": "821e8925e1534d06b7077ca5da7817b7241fe74784965f8ab63cf3ef40728b25",
  "seedHash": "f21cababc3d97ac61c51241c50b897fcad8f7514bdf3269110d5842ec3e227d3",
  "planHash": "11390c307f4d8bd507add26a821a2991dc2f5debb5f48795fb7329f4c7577aa9",
  "candidate": {
    "sourceHash": "94ac077df27ec4687593901405391302b17d2d40752b798ae818dbe9ccbbe949",
    "lockHash": "06a6ba659e9ae2d42ee49f0f11e13d682ed242539c89b6bb455c29cdb089c51f",
    "candidateArtifactHash": "9fba9d6b5a20c906c6c3abeaad247a70b7f2dfef15af965e7600716e65c15f8c"
  }
}