Skip to content
Benchmarks

redact-secret · Report · Detector

Heroku API tokens

  • 29 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

  • Provider documentation HRKU- prefixed OAuth access token (41-character 2024-04 and 65-character 2025-04 generations) · observed 2026-09-23 · the page states "Heroku OAuth access tokens are 65 characters long and prefixed with HRKU-" and shows a 65-character worked example beginning HRKU-AA (not reproduced here, since secret scanners classify it as a live token); its own response examples, changelog-items/2842 and changelog-items/3175 ("increasing from 41 characters to 65 characters") document the earlier 41-character HRKU-<uuid> generation. The literal AA after the dash is example- and tool-observed and the [A-Za-z0-9_-] body alphabet is tool-corroborated; neither is stated in prose
  • gitleaks 8.30.1
  • trufflehog 3.97.4

Re-checked 2026-09-24 (#235, consumed by #209). Heroku documents three generations of the OAuth access token, each "continue[s] to work … until they're regenerated": G0, a bare UUID granted before 2024-04-01 (changelog-items/2842), which is the separate context-gated heroku-api-key-legacy contract; G1, HRKU- plus a UUID, 41 characters, granted 2024-04-01 through 2025-04-22 (changelog-items/2842, 2800, 3175; the oauth article's own response examples and heroku/cli's generated types still show it); and G2, HRKU- plus 60 characters, 65 in total, granted from 2025-04-23 (changelog-items/3175, 3176). An earlier revision of this contract called G1 an "undocumented-width variant"; the provider names 41 as the previous length, so `pattern` now accepts both HRKU- generations. The two pinned tools cover only G2: gitleaks 8.30.1's heroku-api-key-v2 rule (`\b((HRKU-AA[0-9a-zA-Z_-]{58}))`, keyword hrku-aa, entropy 4, trailing delimiter) and trufflehog 3.97.4's heroku/v2 detector both match exactly HRKU-AA plus 58 bytes of [A-Za-z0-9_-]; a G1 token is an expected false negative for both (GitLab's rule, not pinned, covers G1 only). The AA start of G2 is kept in `pattern` as observed grammar (the provider's worked examples and both pinned tools), not as provider-stated grammar: trufflehog#4510 disputes it and osv-scalibr does not pin it, so no fixture asserts silence on a non-AA 65-character value. The authentication article's 40-character hex .netrc example is an undated fourth shape recorded unresolved. Refresh tokens, client secrets, grant codes and session tokens are bare UUIDs and secrets that no family claims; they are never benign controls here. redact-secret#312 (PR #675) froze the G2 grammar only on the product side.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of Heroku API tokens fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated12False alarmsat most 24.3%0 of 12 controls flagged12 quiet · 0 flagged
  • flare-redact at most 24.3%
  • gitleaks at most 24.3%
  • trufflehog at most 35.4%
Must not flag · Project policy5False alarmsat most 43.4%0 of 5 controls flagged5 quiet · 0 flagged
  • flare-redact at most 43.4%
  • gitleaks at most 43.4%
  • trufflehog at most 43.4%
Must redact · Provider-documented12Secret spans left readableat most 22.8%0 of 13 spansNear-twins told apartat least 70.1%9 of 9 pairs13 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 98.6%
  • gitleaks at most 33.3%
  • trufflehog at most 42.2%
29 of 29 rows
Fixtures
29
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

29 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in Heroku API tokens
FixtureKind and evidenceredact-secret
heroku-api-key-authorization-ids-public-idheroku · public-identifierMust not flagT2 · Tool-corroboratedQuiet
heroku-api-key-authorizations-create-g1heroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-compose-envheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-env-g1heroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-github-actions-deployheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-netrc-g2heroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-prefixed-shape-bareheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-prefixed-shape-quotedheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-prefixed-shape-unicode-crlfheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-python-heroku3-clientheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-shell-export-api-keyheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-terraform-providerheroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-authorizations-create-g1-separator-twinheroku · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-bearer-header-g2heroku · documented-format-literalMust redactT1 · Provider-documentedRedacted
heroku-api-key-bearer-header-g2-prefix-case-twinheroku · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-env-g1-length-twinheroku · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-generation-note-proseheroku · prose-mentionMust not flagT3 · Project policyQuiet
heroku-api-key-label-proseheroku · benign-lookalikeMust not flagT3 · Project policyQuiet
heroku-api-key-maskheroku · benign-lookalikeMust not flagT3 · Project policyQuiet
heroku-api-key-netrc-template-placeholderheroku · documentation-placeholderMust not flagT3 · Project policyQuiet
heroku-api-key-prefix-onlyheroku · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
heroku-api-key-prefixed-shape-bare-twinheroku · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-prefixed-shape-prefix-bare-twinheroku · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-prefixed-shape-prefix-quoted-twinheroku · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-prefixed-shape-prefix-unicode-crlf-twinheroku · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-prefixed-shape-quoted-twinheroku · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-prefixed-shape-unicode-crlf-twinheroku · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
heroku-api-key-referenceheroku · benign-lookalikeMust not flagT3 · Project policyQuiet
heroku-api-key-short-bodyheroku · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet