Skip to content
Benchmarks

redact-secret · Report · Detector

AI21 API keys

  • 60 fixtures
  • Format evidence: T2 · Tool-corroborated
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

Context-gated family (#384, product redact-secret#868; research #784, T2 by redact-secret#1013). No AI21 page, staff statement or SDK code states a shape, so nothing reaches T1. The redact-secret#1013 pass (frozen at redact-secret add1188f) found the corroborated route without any provider material: the Vulnetix vnx-sec-315 peer rule (an ai21 key name then exactly 32 alphanumerics) and two independent implementations (keychecker, LLM-Runner-Router) that accept exactly [A-Za-z0-9]{32}: 3 references, 3 owners, 2 non-summary classes. AI21-committed key literals in its public example code (six, each 32 mixed alphanumerics, none a UUID) are of unknown validity; whether they count as provider examples is maintainer ruling Q-AI, so they are not counted and appear only as bounded evidence in the empirical record. Kingfisher (2025-07 to its 2026-08 deletion) read a UUID and octocode reads 40-64 alphanumerics; the contract claims exactly 32 and excludes both. The value is recognised only beside a same-line AI21_API_KEY name, an api.ai21.com host or an AI21 SDK constructor, so its positives score as project policy and its twins keep the value and change only the context; no bare-value claim is made, since a bare 32-character run collides with hashes, UUID fragments and other providers' keys.

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of AI21 API keys fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated18False alarmsat most 17.6%0 of 18 controls flagged18 quiet · 0 flagged
  • flare-redact at most 17.6%
  • gitleaks at most 17.6%
  • trufflehog at most 17.6%
Must not flag · Project policy25False alarmsat most 13.3%0 of 25 controls flagged25 quiet · 0 flagged
  • flare-redact at most 13.3%
  • gitleaks at most 19.5%
  • trufflehog at most 13.3%
Project policy · Project policy17Secret spans left readableat most 18.4%0 of 17 spansNear-twins told apartat least 81.6%17 of 17 pairs17 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 99.0%
  • gitleaks at most 18.4%
  • trufflehog at most 100.0%
60 of 60 rows
Fixtures
60
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

60 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in AI21 API keys
FixtureKind and evidenceredact-secret
ai21-api-key-actions-env-literalai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-compose-envai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-curl-headerai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-docker-run-envai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-dotenvai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-dotenv-altai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-exportai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-json-configai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-key-shape-bareai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-key-shape-quotedai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-key-shape-unicode-crlfai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-langchain-kwargai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-litellm-yamlai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-other-host-curl-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-proxy-logai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-python-ctorai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-tool-callai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-ts-ctorai21 · documented-format-literalProject policyT3 · Project policyRedacted
ai21-api-key-actions-secret-referenceai21 · templated-referenceMust not flagT3 · Project policyQuiet
ai21-api-key-angle-key-placeholderai21 · documentation-placeholderMust not flagT3 · Project policyQuiet
ai21-api-key-bare-in-prose-near-missai21 · format-near-missMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-bare-line-near-missai21 · format-near-missMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-base64-text-encoded-valueai21 · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-data-uri-encoded-valueai21 · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-docs-ctor-placeholderai21 · documentation-placeholderMust not flagT3 · Project policyQuiet
ai21-api-key-embedded-run-near-missai21 · format-near-missMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-env-reference-referenceai21 · templated-referenceMust not flagT3 · Project policyQuiet
ai21-api-key-id-named-compose-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-id-named-env-alt-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-id-named-env-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-id-named-export-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-id-named-json-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-id-named-kwarg-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-id-named-tool-call-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-key-guidance-proseai21 · prose-mentionMust not flagT3 · Project policyQuiet
ai21-api-key-key-shape-bare-twinai21 · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
ai21-api-key-key-shape-quoted-twinai21 · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
ai21-api-key-key-shape-unicode-crlf-twinai21 · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
ai21-api-key-label-proseai21 · benign-lookalikeMust not flagT3 · Project policyQuiet
ai21-api-key-long-value-twinai21 · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
ai21-api-key-maskai21 · benign-lookalikeMust not flagT3 · Project policyQuiet
ai21-api-key-missing-keywordai21 · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-model-id-public-idai21 · public-identifierMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-model-notes-proseai21 · prose-mentionMust not flagT3 · Project policyQuiet
ai21-api-key-object-id-public-idai21 · public-identifierMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-other-host-log-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-other-provider-model-twinai21 · missing-context-markerMust not flagT3 · Project policy · twinQuiet
ai21-api-key-project-uuid-public-idai21 · public-identifierMust not flagT2 · Tool-corroboratedQuiet
ai21-api-key-python-environ-referenceai21 · templated-referenceMust not flagT3 · Project policyQuiet
ai21-api-key-referenceai21 · benign-lookalikeMust not flagT3 · Project policyQuiet