Skip to content
Benchmarks

redact-secret · Report · Detector

Generic tokens

  • 237 fixtures
  • Format evidence: T3 · Project policy
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

  • Twin source context-twin decision (#36) · observed 2026-09-20 · no provider exists for an arbitrary literal. The recorded decision is that the twin keeps the value and mutates exactly one property of the assignment context; silence is project policy, never a format claim

An arbitrary literal in a sensitive field is a masking-policy case, not a provider-format ground truth.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of Generic tokens fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Provider-documented6False alarmsat most 39.0%0 of 6 controls flagged6 quiet · 0 flagged
  • flare-redact at most 39.0%
  • gitleaks at most 39.0%
  • trufflehog at most 39.0%
Must not flag · Project policy100False alarmsat most 5.4%1 of 100 controls flagged99 quiet · 1 flagged
  • flare-redact at most 5.4%
  • gitleaks at most 3.7%
  • trufflehog at most 3.7%
Must redact · Provider-documented3Secret spans left readableinsufficient-evidence1 of 3 spans2 redacted · 0 too much · 0 partly exposed · 1 missed
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Project policy · Project policy112Secret spans left readableat most 39.4%34 of 112 spansNear-twins told apartat least 72.2%10 of 10 pairs78 redacted · 0 too much · 0 partly exposed · 34 missed
  • flare-redact at most 99.8%
  • gitleaks at most 61.7%
  • trufflehog at most 100.0%
Pending review16UnscoredNot scoredInspect only: never scored until evidence existsNot scored
237 of 237 rows
Fixtures
237
Left readable
35
Redacted too much
0
False alarms
1

Fixtures for this detector

237 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in Generic tokens
FixtureKind and evidenceredact-secret
generic-literal-alphabet-edge-base64-standard-padded-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-standard-padded-in-jsonbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-standard-padded-in-urlbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-standard-padded-three-layers-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-standard-unpadded-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-url-safe-padded-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-url-safe-unpadded-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-url-safe-unpadded-in-jsonbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-url-safe-unpadded-in-urlbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-url-safe-unpadded-of-base64-standard-padded-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-base64-url-safe-unpadded-of-hex-upper-of-base64-standard-padded-in-jsonbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-lower-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-lower-in-urlbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-lower-of-base64-standard-padded-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-lower-of-hex-lower-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-mixed-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-upper-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-alphabet-edge-hex-upper-in-jsonbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-standard-padded-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-standard-padded-in-jsonbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-standard-padded-in-urlbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-standard-padded-three-layers-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-url-safe-unpadded-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-url-safe-unpadded-in-jsonbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-url-safe-unpadded-in-urlbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-url-safe-unpadded-of-base64-standard-padded-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-base64-url-safe-unpadded-of-hex-upper-of-base64-standard-padded-in-jsonbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-lower-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-lower-in-urlbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-lower-of-base64-standard-padded-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-lower-of-hex-lower-in-envbase64-hex-representation-projections · credential-in-nested-encoding-layersProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-mixed-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-upper-in-envbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
generic-literal-padding-edge-hex-upper-in-jsonbase64-hex-representation-projections · credential-in-base64-or-hex-formProject policyT3 · Project policyLeft readable
state-json-output-password-with-sensitive-truehashicorp-terraform-authored · terraform-json-sensitive-attribute-literal-beside-true-flagMust redactT1 · Provider-documentedLeft readable
stdout-mask-where-source-has-environment-referencejupyter-notebook-files-authored · already-masked-valuesMust not flagT3 · Project policyFlagged
generic-literal-alphabet-edge-basebase64-hex-representation-bases · literal-secrets-under-generic-credential-namesProject policyT3 · Project policyRedacted
generic-literal-padding-edge-basebase64-hex-representation-bases · literal-secrets-under-generic-credential-namesProject policyT3 · Project policyRedacted
api-keygeneric · literal-secrets-under-generic-credential-namesProject policyT3 · Project policyRedacted
arn-accountgeneric · secret-manager-grammar-imitationsProject policyT3 · Project policyRedacted
authgeneric · nested-yaml-literal-passwordsProject policyT3 · Project policyRedacted
aws-secret-access-key-truncated-near-missgeneric · credential-named-literal-near-missProject policyT3 · Project policyRedacted
block-literalgeneric · block-yaml-and-brace-prefixed-secret-literalsProject policyT3 · Project policyRedacted
browserbase-api-key-bb-test-key-near-missgeneric · credential-named-literal-near-missProject policyT3 · Project policyRedacted
cerebras-api-key-pinecone-hyphen-key-near-missgeneric · credential-named-literal-near-missProject policyT3 · Project policyRedacted
changed-secretgeneric · aws-example-keys-one-character-offProject policyT3 · Project policyRedacted
client-secretgeneric · literal-secrets-under-generic-credential-namesProject policyT3 · Project policyRedacted
confluent-cloud-api-secret-legacy-overlong-value-near-missgeneric · credential-named-literal-near-missProject policyT3 · Project policyRedacted
databasegeneric · nested-yaml-literal-passwordsProject policyT3 · Project policyRedacted
datadog-api-key-short-keygeneric · documented-format-literalProject policyT3 · Project policyRedacted