Skip to content
Benchmarks

redact-secret · Report · Detector

RubyGems API keys

  • 39 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

Arrival evidence (#528, product redact-secret#1023; #1014 handoff rubygems.md, READY). A key can push and yank gems and manage owners within its scopes, so a leak is a supply-chain risk for every gem the account owns. T1 for the prefix, the 48-byte length and the lowercase-hex alphabet under R1, from the provider's own generator; OIDC-exchanged keys share it. RubyGems sends the key in Authorization without a scheme. Excluded: legacy unprefixed keys (no shape; generic context covers GEM_HOST_API_KEY= and :rubygems_api_key:), rubygems_version and rubygems_mfa_required metadata keys (they fail the body) and uppercased copies. gitleaks 8.30.1 agrees with the grammar but also requires a trailing delimiter; trufflehog 3.97.4 reads a wider class (any lowercase letter and A), so it overreaches on a non-hex letter twin. Graduated to a registry detector at the 4fb7882 re-pin (redact-secret PR #1039).

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of RubyGems API keys fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated15False alarmsat most 20.4%0 of 15 controls flagged15 quiet · 0 flagged
  • flare-redact at most 20.4%
  • gitleaks at most 20.4%
  • trufflehog at most 29.8%
Must not flag · Project policy8False alarmsat most 32.4%0 of 8 controls flagged8 quiet · 0 flagged
  • flare-redact at most 32.4%
  • gitleaks at most 32.4%
  • trufflehog at most 32.4%
Must redact · Provider-documented16Secret spans left readableat most 19.4%0 of 16 spansNear-twins told apartat least 70.1%9 of 9 pairs16 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 98.9%
  • gitleaks at most 19.4%
  • trufflehog at most 19.4%
39 of 39 rows
Fixtures
39
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

39 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in RubyGems API keys
FixtureKind and evidenceredact-secret
rubygems-api-key-actions-envrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-authorization-no-schemerubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-bare-proserubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-bearer-headerrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-chat-pasterubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-dotenvrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-exportrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-gem-credentials-yamlrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-gem-push-envrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-json-api-keyrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-json-tokenrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-key-shape-barerubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-key-shape-quotedrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-key-shape-unicode-crlfrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-non-hex-letter-twinrubygems · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-sdk-kwargrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-x-api-key-headerrubygems · documented-format-literalMust redactT1 · Provider-documentedRedacted
rubygems-api-key-actions-secret-referencerubygems · templated-referenceMust not flagT3 · Project policyQuiet
rubygems-api-key-bare-48-hex-encoded-valuerubygems · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
rubygems-api-key-body-47-twinrubygems · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-body-49-twinrubygems · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-credentials-placeholder-placeholderrubygems · documentation-placeholderMust not flagT3 · Project policyQuiet
rubygems-api-key-ellipsis-placeholderrubygems · documentation-placeholderMust not flagT3 · Project policyQuiet
rubygems-api-key-env-reference-referencerubygems · templated-referenceMust not flagT3 · Project policyQuiet
rubygems-api-key-gemspec-metadata-public-idrubygems · public-identifierMust not flagT2 · Tool-corroboratedQuiet
rubygems-api-key-hyphen-separator-twinrubygems · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-key-guidance-proserubygems · prose-mentionMust not flagT3 · Project policyQuiet
rubygems-api-key-label-proserubygems · benign-lookalikeMust not flagT3 · Project policyQuiet
rubygems-api-key-leading-glue-twinrubygems · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-lockfile-version-public-idrubygems · public-identifierMust not flagT2 · Tool-corroboratedQuiet
rubygems-api-key-maskrubygems · benign-lookalikeMust not flagT3 · Project policyQuiet
rubygems-api-key-prefix-onlyrubygems · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
rubygems-api-key-referencerubygems · benign-lookalikeMust not flagT3 · Project policyQuiet
rubygems-api-key-short-bodyrubygems · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
rubygems-api-key-trailing-letter-twinrubygems · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-trailing-underscore-twinrubygems · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-truncated-near-missrubygems · format-near-missMust not flagT2 · Tool-corroboratedQuiet
rubygems-api-key-uppercase-hex-byte-twinrubygems · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
rubygems-api-key-uppercase-prefix-twinrubygems · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet