Skip to content
Benchmarks

redact-secret · Report · Detector

Notion tokens

  • 29 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

  • Provider documentation 2024-09 Public API token prefix migration · observed 2026-09-23 · the provider documents secret_ as the still-working legacy Public API token prefix and ntn_ as the prefix for newly generated tokens; the 43-character body and alphabet remain tool-corroborated
  • trufflehog 3.97.4

This contract covers only the legacy secret_ family. Notion says the token should be treated as opaque, warns that its format may change, and advises against regex validation. The current ntn_ family is distinct and remains outside this contract; the pinned gitleaks rule targets ntn_, while TruffleHog corroborates secret_.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of Notion tokens fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated13False alarmsat most 22.8%0 of 13 controls flagged13 quiet · 0 flagged
  • flare-redact at most 22.8%
  • gitleaks at most 22.8%
  • trufflehog at most 22.8%
Must not flag · Project policy4False alarmsinsufficient-evidence0 of 4 controls flagged4 quiet · 0 flagged
  • flare-redact insufficient-evidence
  • gitleaks insufficient-evidence
  • trufflehog insufficient-evidence
Must redact · Provider-documented12Secret spans left readableat most 24.3%0 of 12 spansNear-twins told apartat least 70.1%9 of 9 pairs12 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 24.3%
  • gitleaks at most 60.9%
  • trufflehog at most 53.2%
29 of 29 rows
Fixtures
29
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

29 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in Notion tokens
FixtureKind and evidenceredact-secret
notion-token-bearer-headernotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-shape-1-barenotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-shape-1-quotednotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-shape-1-unicode-crlfnotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-bearer-header-separator-twinnotion · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-compose-envnotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-curl-bearernotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-database-and-page-ids-public-idnotion · public-identifierMust not flagT2 · Tool-corroboratedQuiet
notion-token-debug-client-lognotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-env-api-keynotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-env-api-key-length-twinnotion · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-github-actions-envnotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-label-prosenotion · benign-lookalikeMust not flagT3 · Project policyQuiet
notion-token-masknotion · benign-lookalikeMust not flagT3 · Project policyQuiet
notion-token-mcp-server-envnotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-prefix-onlynotion · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
notion-token-referencenotion · benign-lookalikeMust not flagT3 · Project policyQuiet
notion-token-sdk-clientnotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-sdk-client-prefix-case-twinnotion · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shape-1-bare-twinnotion · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shape-1-prefix-bare-twinnotion · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shape-1-prefix-quoted-twinnotion · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shape-1-prefix-unicode-crlf-twinnotion · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shape-1-quoted-twinnotion · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shape-1-unicode-crlf-twinnotion · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
notion-token-shell-export-tokennotion · documented-format-literalMust redactT1 · Provider-documentedRedacted
notion-token-short-bodynotion · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
notion-token-snake-case-identifier-near-missnotion · format-near-missMust not flagT2 · Tool-corroboratedQuiet
notion-token-vault-note-prosenotion · prose-mentionMust not flagT3 · Project policyQuiet