EVALUATION METHOD
Benign
Controls that resemble secrets but are not: placeholders, references, near misses, public identifiers, encoded values and ordinary text. None is transformed. Each should produce no finding.
1 · How it runs
Does the scanner leave harmless look-alikes alone?
- InputA control from a named taxonomy, read as authored.
- ChangeNone. The control is not transformed.
- CheckThe scanner reports nothing on it. A control is counted once per scanner, however many findings it gets.
- Controls
- 2,277
- Taxonomies
- 12
- Suites
- 69
2 · Recorded now
Which controls were flagged?
Controls by taxonomy, per scanner. Findings are not counted, only whether a control was flagged.
| Taxonomy | redact-secret0.1.0-beta.14 | gitleaks8.30.1 | trufflehog3.97.4 | flare-redact1.6.1 |
|---|---|---|---|---|
| Controls by family axis | ||||
| Encoded value | 1 of 147 | 19 of 147 | 1 of 147 | 2 of 147 |
| Near miss | 0 of 515 | 14 of 515 | 7 of 515 | 15 of 515 |
| Ordinary prose | 0 of 176 | 0 of 176 | 0 of 176 | 2 of 176 |
| Placeholder | 3 of 551 | 12 of 551 | 11 of 551 | 23 of 551 |
| Public identifier | 0 of 349 | 17 of 349 | 11 of 349 | 15 of 349 |
| Reference | 0 of 419 | 0 of 419 | 0 of 419 | 2 of 419 |
| Real-world shapes, untargeted | ||||
| Agent output | 0 of 20 | 0 of 20 | 0 of 20 | 0 of 20 |
| Config | 0 of 20 | 1 of 20 | 1 of 20 | 1 of 20 |
| Docs | 0 of 20 | 0 of 20 | 0 of 20 | 1 of 20 |
| Lockfile | 0 of 20 | 1 of 20 | 0 of 20 | 0 of 20 |
| Logs | 0 of 20 | 1 of 20 | 0 of 20 | 0 of 20 |
| Source | 1 of 20 | 1 of 20 | 0 of 20 | 1 of 20 |
| Untargeted, by policy action | ||||
| Flagged with redact or blockOnly a scanner that reports a policy action has a count hereNot listed: these controls are the untargeted taxonomy rows above, split by the action a finding carried. Open a count there. | 0 of 120 | No check | No check | No check |
| Flagged with warn onlyAccepted by the product policy on ordinary proseNot listed: these controls are the untargeted taxonomy rows above, split by the action a finding carried. Open a count there. | 1 of 120 | No check | No check | No check |
Controls flagged, of the controls in that taxonomy.
A count opens the checks behind it: the ones that did not hold, from this run only. "Needs review" opens the checks that wait for a person. A zero has none to open, and a scanner that did not run has none to list.
3 · How to read it
Reading benign
- The taxonomy rows split one set of controls, so read a number inside its row. They are not added up here.
- Real-world-shaped controls are measured apart from every family: a flag there is discovery evidence about ordinary content, not about a family. A warn-only finding on ordinary prose is accepted by the product policy, so the last group separates the actions that gate from the ones that only warn.
4 · Exact inputs
Where the cases come from
2,277 controls read by this method. Synthetic content only.
Suites
Every case comes from a published suite. Open a suite to read its fixtures.