Skip to content
Benchmarks

redact-secret · Report · Detector

Anthropic tokens

  • 30 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

The same provider table assigns sk-ant-api01- to Compliance Access Keys and sk-ant-admin01- to Admin API keys. They are real sibling credential classes, not members of this contract, and back prefix twins. The provider does not document this contract's body length, alphabet or AA suffix.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of Anthropic tokens fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Provider-documented2False alarmsinsufficient-evidence0 of 2 controls flagged2 quiet · 0 flagged
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Must not flag · Tool-corroborated9False alarmsat most 29.9%0 of 9 controls flagged9 quiet · 0 flagged
  • flare-redact at most 64.6%
  • gitleaks at most 43.5%
  • trufflehog at most 29.9%
Must not flag · Project policy3False alarmsinsufficient-evidence0 of 3 controls flagged3 quiet · 0 flagged
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Must redact · Provider-documented9Secret spans left readableat most 29.9%0 of 9 spansNear-twins told apartat least 61.0%6 of 6 pairs9 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 29.9%
  • gitleaks at most 29.9%
  • trufflehog at most 43.5%
Project policy · Project policy3Secret spans left readableinsufficient-evidence0 of 3 spans3 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Pending review4UnscoredNot scoredInspect only: never scored until evidence existsNot scored
30 of 30 rows
Fixtures
30
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

30 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in Anthropic tokens
FixtureKind and evidenceredact-secret
anthropic-token-api03-no-final-hyphen-prefix-twinanthropic · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
anthropic-token-api03-plain-twinanthropic · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
anthropic-token-api03-unicode-crlf-twinanthropic · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
anthropic-token-api03-vendor-stem-prefix-twinanthropic · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
anthropic-token-python-clientanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-shape-1-bareanthropic · documented-format-literalProject policyT3 · Project policyRedacted
anthropic-token-shape-1-quotedanthropic · documented-format-literalProject policyT3 · Project policyRedacted
anthropic-token-shape-1-unicode-crlfanthropic · documented-format-literalProject policyT3 · Project policyRedacted
anthropic-token-api03-admin-prefix-plain-twinanthropic · unsettled-evidence-inputPending reviewT0 · Pending · twinUnscored
anthropic-token-api03-admin-prefix-unicode-crlf-twinanthropic · unsettled-evidence-inputPending reviewT0 · Pending · twinUnscored
anthropic-token-api03-compliance-prefix-plain-twinanthropic · unsettled-evidence-inputPending reviewT0 · Pending · twinUnscored
anthropic-token-api03-compliance-prefix-unicode-crlf-twinanthropic · unsettled-evidence-inputPending reviewT0 · Pending · twinUnscored
anthropic-token-api03-curl-x-api-keyanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-api03-dotenvanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-api03-plainanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-api03-python-sdkanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-api03-underscore-prefix-twinanthropic · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
anthropic-token-api03-unicode-crlfanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-env-api-keyanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-env-template-placeholderanthropic · documentation-placeholderMust not flagT1 · Provider-documentedQuiet
anthropic-token-key-classes-note-proseanthropic · prose-mentionMust not flagT3 · Project policyQuiet
anthropic-token-label-proseanthropic · benign-lookalikeMust not flagT1 · Provider-documentedQuiet
anthropic-token-maskanthropic · benign-lookalikeMust not flagT3 · Project policyQuiet
anthropic-token-mcp-server-envanthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-prefix-onlyanthropic · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
anthropic-token-referenceanthropic · benign-lookalikeMust not flagT3 · Project policyQuiet
anthropic-token-response-identifiers-public-idanthropic · public-identifierMust not flagT2 · Tool-corroboratedQuiet
anthropic-token-short-bodyanthropic · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
anthropic-token-x-api-key-headeranthropic · documented-format-literalMust redactT1 · Provider-documentedRedacted
anthropic-token-x-api-key-header-stem-twinanthropic · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet