Skip to content
Benchmarks

redact-secret · Report · Detector

Vault tokens

  • 28 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

  • Provider documentation hvs./hvb./hvr. prefixes, 24+ random characters · observed 2026-09-17 · prefixes and minimum length; the provider states the structure is opaque, so the 90–120-character rule the pinned tools use is corroboration, not contract
  • trufflehog 3.97.4
  • gitleaks 8.30.1

Re-checked 2026-09-20 (#46): the same page documents exactly three prefixes (hvs./hvb./hvr.), all already in this contract's pattern; the prefix twin (hvx.) is a single-character deviation from the documented set rather than a real alternate token type, since no fourth Vault token prefix exists to cite.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of Vault tokens fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated10False alarmsat most 27.8%0 of 10 controls flagged10 quiet · 0 flagged
  • flare-redact at most 51.0%
  • gitleaks at most 27.8%
  • trufflehog at most 27.8%
Must not flag · Project policy4False alarmsinsufficient-evidence0 of 4 controls flagged4 quiet · 0 flagged
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Must redact · Provider-documented2Secret spans left readableinsufficient-evidence0 of 2 spansNear-twins told apartat least 61.0%6 of 6 pairs2 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Project policy · Project policy12Secret spans left readableat most 24.3%0 of 12 spans12 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 24.3%
  • gitleaks at most 91.1%
  • trufflehog at most 95.3%
28 of 28 rows
Fixtures
28
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

28 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in Vault tokens
FixtureKind and evidenceredact-secret
vault-token-service-plain-twinhashicorp-vault · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
vault-token-service-unicode-crlf-twinhashicorp-vault · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
vault-token-shape-1-barehashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-1-quotedhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-1-unicode-crlfhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-2-barehashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-2-quotedhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-2-unicode-crlfhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-3-barehashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-3-quotedhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-shape-3-unicode-crlfhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-x-vault-token-headerhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-audit-hmac-encoded-valuehashicorp-vault · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
vault-token-env-vault-tokenhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted
vault-token-label-prosehashicorp-vault · benign-lookalikeMust not flagT3 · Project policyQuiet
vault-token-maskhashicorp-vault · benign-lookalikeMust not flagT3 · Project policyQuiet
vault-token-prefix-onlyhashicorp-vault · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
vault-token-referencehashicorp-vault · benign-lookalikeMust not flagT3 · Project policyQuiet
vault-token-service-boundary-plain-twinhashicorp-vault · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
vault-token-service-boundary-unicode-crlf-twinhashicorp-vault · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
vault-token-service-plainhashicorp-vault · documented-format-literalMust redactT1 · Provider-documentedRedacted
vault-token-service-prefix-plain-twinhashicorp-vault · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
vault-token-service-prefix-unicode-crlf-twinhashicorp-vault · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
vault-token-service-unicode-crlfhashicorp-vault · documented-format-literalMust redactT1 · Provider-documentedRedacted
vault-token-short-bodyhashicorp-vault · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
vault-token-token-lookup-ids-public-idhashicorp-vault · public-identifierMust not flagT2 · Tool-corroboratedQuiet
vault-token-token-types-note-prosehashicorp-vault · prose-mentionMust not flagT3 · Project policyQuiet
vault-token-vault-loginhashicorp-vault · documented-format-literalProject policyT3 · Project policyRedacted