Skip to content
Benchmarks

redact-secret · Report · Detector

AWS access keys

  • 31 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

  • Provider documentation IAM unique-ID prefix table · observed 2026-09-17 · AKIA/ASIA/ABIA/ACCA prefixes, and AIDA as the IAM-user unique-ID prefix rather than an access key (re-checked 2026-09-20, #36); 16-character base32 body and 40-character secret are tool-corroborated
  • trufflehog 3.97.4
  • gitleaks 8.30.1

A separate 40-character secret access key is required. ASIA additionally needs a session token and is not covered by this contract.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of AWS access keys fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated9False alarmsat most 29.9%0 of 9 controls flagged9 quiet · 0 flagged
  • flare-redact at most 87.9%
  • gitleaks at most 29.9%
  • trufflehog at most 29.9%
Must not flag · Project policy5False alarmsat most 43.4%0 of 5 controls flagged5 quiet · 0 flagged
  • flare-redact at most 76.9%
  • gitleaks at most 43.4%
  • trufflehog at most 43.4%
Must redact · Provider-documented2Secret spans left readableinsufficient-evidence0 of 4 spans4 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Project policy · Project policy14Secret spans left readableat most 21.5%0 of 14 spansNear-twins told apartat least 61.0%6 of 6 pairs14 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 21.5%
  • gitleaks at most 78.6%
  • trufflehog at most 100.0%
Pending review1UnscoredNot scoredInspect only: never scored until evidence existsNot scored
31 of 31 rows
Fixtures
31
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

31 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in AWS access keys
FixtureKind and evidenceredact-secret
aws-idauthored-aws · early-filler-token-assignmentsProject policyT3 · Project policyRedacted
aws-access-key-boto3-clientaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-configure-credentials-actionaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-1-bareaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-1-bare-twinaws · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
aws-access-key-shape-1-quotedaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-1-quoted-twinaws · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
aws-access-key-shape-1-unicode-crlfaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-1-unicode-crlf-twinaws · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
aws-access-key-shape-2-bareaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-2-bare-twinaws · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
aws-access-key-shape-2-quotedaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-2-quoted-twinaws · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
aws-access-key-shape-2-unicode-crlfaws · documented-format-literalProject policyT3 · Project policyRedacted
aws-access-key-shape-2-unicode-crlf-twinaws · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
changed-idaws · aws-example-keys-one-character-offProject policyT3 · Project policyRedacted
id-bareaws · aws-documented-example-keysMust not flagT3 · Project policyQuiet
id-envaws · aws-documented-example-keysMust not flagT3 · Project policyQuiet
aws-key-id-shaped-body-of-example-marker-basepolicy-ambiguous-assignment-bases-authored · example-marker-inside-provider-shaped-value-not-published-by-providerProject policyT3 · Project policyRedacted
aws-key-id-shaped-body-of-example-marker-in-english-prosepolicy-ambiguous-assignment-projections-generated · example-marker-inside-provider-shaped-value-not-published-by-providerProject policyT3 · Project policyRedacted
aws-key-id-shaped-body-of-example-marker-in-korean-prosepolicy-ambiguous-assignment-projections-generated · example-marker-inside-provider-shaped-value-not-published-by-providerProject policyT3 · Project policyRedacted
aws-key-id-shaped-body-of-example-marker-in-markdown-fencepolicy-ambiguous-assignment-projections-generated · example-marker-inside-provider-shaped-value-not-published-by-providerProject policyT3 · Project policyRedacted
aws-access-key-iam-user-identity-public-idaws · public-identifierMust not flagT2 · Tool-corroboratedQuiet
aws-access-key-maskaws · benign-lookalikeMust not flagT3 · Project policyQuiet
aws-access-key-pair-plainaws · documented-format-literalMust redactT1 · Provider-documentedRedacted
aws-access-key-pair-unicode-crlfaws · documented-format-literalMust redactT1 · Provider-documentedRedacted
aws-access-key-prefix-onlyaws · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
aws-access-key-referenceaws · benign-lookalikeMust not flagT3 · Project policyQuiet
aws-access-key-rotation-note-proseaws · prose-mentionMust not flagT3 · Project policyQuiet
aws-access-key-short-bodyaws · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
identifier-aloneaws-compound-credentials-authored · aws-access-key-id-without-secret-confidentialityPending reviewT0 · PendingUnscored