Skip to content
Benchmarks

redact-secret · Report · Detector

Composio API keys

  • 48 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

Arrival evidence (#434, product redact-secret#909; #860 handoff composio.md, READY). The project key has full access to one project and its connected third-party accounts. T1 on every fact: documented prefix, a dated provider-staff statement (R3) for the width and alphabet, the OpenAPI example for the width and the CLI redaction regex for the alphabet (R2). The short ak_ prefix with a _/- alphabet would match snake_case and kebab-case identifiers, so the contract carries the handoff's mixed-case guard (at least one upper- and one lower-case letter; ~6e-5 of random bodies fail it, an accepted false negative), and all-lower/all-upper bodies back twins. A mixed-case 20-byte identifier after ak_ remains an accepted false positive. bkend.ai's ak_ + 64 hex is another provider's credential and a twin, not a control. Neither pinned peer has a Composio rule. Graduated to a registry detector at the 1127bf9 re-pin (redact-secret PR #938).

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of Composio API keys fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated21False alarmsat most 15.5%0 of 21 controls flagged21 quiet · 0 flagged
  • flare-redact at most 15.5%
  • gitleaks at most 28.9%
  • trufflehog at most 15.5%
Must not flag · Project policy8False alarmsat most 32.4%0 of 8 controls flagged8 quiet · 0 flagged
  • flare-redact at most 32.4%
  • gitleaks at most 32.4%
  • trufflehog at most 32.4%
Must redact · Provider-documented19Secret spans left readableat most 16.8%0 of 19 spansNear-twins told apartat least 78.5%14 of 14 pairs19 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 99.1%
  • gitleaks at most 59.0%
  • trufflehog at most 100.0%
48 of 48 rows
Fixtures
48
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

48 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in Composio API keys
FixtureKind and evidenceredact-secret
composio-api-key-actions-envcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-bare-prosecomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-bearer-headercomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-chat-pastecomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-compose-envcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-csv-commacomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-digest-encoded-valuecomposio · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-dotenvcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-end-of-inputcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-exportcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-inside-xak-twincomposio · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-json-api-keycomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-json-tokencomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-key-shape-barecomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-key-shape-quotedcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-key-shape-unicode-crlfcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-mcp-configcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-provider-headercomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-sdk-kwargcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-ts-constructorcomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-x-api-key-headercomposio · documented-format-literalMust redactT1 · Provider-documentedRedacted
composio-api-key-actions-secret-referencecomposio · templated-referenceMust not flagT3 · Project policyQuiet
composio-api-key-all-lowercase-body-twincomposio · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-all-uppercase-body-twincomposio · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-bkend-ak-64-hex-twincomposio · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-body-long-twincomposio · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-body-short-twincomposio · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-dot-in-body-twincomposio · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-ellipsis-placeholdercomposio · documentation-placeholderMust not flagT3 · Project policyQuiet
composio-api-key-env-reference-referencecomposio · templated-referenceMust not flagT3 · Project policyQuiet
composio-api-key-hyphen-prefix-twincomposio · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-inside-cak-twincomposio · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-inside-oak-twincomposio · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-inside-uak-twincomposio · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-kebab-case-identifier-near-misscomposio · format-near-missMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-key-guidance-prosecomposio · prose-mentionMust not flagT3 · Project policyQuiet
composio-api-key-label-prosecomposio · benign-lookalikeMust not flagT3 · Project policyQuiet
composio-api-key-maskcomposio · benign-lookalikeMust not flagT3 · Project policyQuiet
composio-api-key-plus-in-body-twincomposio · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-prefix-onlycomposio · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-project-id-public-idcomposio · public-identifierMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-referencecomposio · benign-lookalikeMust not flagT3 · Project policyQuiet
composio-api-key-short-bodycomposio · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-sibling-placeholders-placeholdercomposio · documentation-placeholderMust not flagT3 · Project policyQuiet
composio-api-key-snake-case-identifier-near-misscomposio · format-near-missMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-trailing-glue-twincomposio · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
composio-api-key-truncated-near-misscomposio · format-near-missMust not flagT2 · Tool-corroboratedQuiet
composio-api-key-uppercase-prefix-twincomposio · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet