Skip to content
Benchmarks

redact-secret · Report · Detector

ClickHouse Cloud API secrets

  • 45 fixtures
  • Format evidence: T1 · Provider-documented
  • Run 2026-10-07
  • Mode published · redact-secret 0.1.0-beta.14

Format evidence

Arrival evidence (#464, product redact-secret#971; #860 handoff clickhouse-cloud.md, READY). T1 as of 2025-04-16: a ClickHouse employee stated the 4b1d prefix in gitleaks PR #1826 (R3) and authored the rule 4b1d[A-Za-z0-9]{38} (R2), and the provider-owned Terraform examples and a 2024 unit-test fixture carry 42-byte 4b1d values. One provider knowledge-base example is 39 bytes (2023-09-02); R3 date order sets it aside, so a 39-byte total is a twin, not a positive. POLICY, not T1: the product requires at least one uppercase letter in the body (removes hex digests and lowercase ids that start 4b1d; false-negative cost about 1e-9), and the staff regex does not, so every positive is mixed case and no fixture asserts silence on a lowercase body. The leading boundary matters because 4b1d is valid hex: a longer hex or base64 run containing 4b1d must not match mid-run. Excluded: the key ID (no prefix, 17 or 20 alphanumerics, unresolved), UUIDs containing -4b1d-, 40- and 64-hex digests starting 4b1d, secrets supplied through the API's hashData (caller-chosen shape) and placeholders such as mykeysecret. Graduated to a registry detector at the 4fb7882 re-pin (redact-secret PR #1037).

What the run recorded, by group

Detector views overlap, so their groups are never summed across detectors. Other scanners are reference values on the same inputs, in run order.
Groups of ClickHouse Cloud API secrets fixtures
GroupFixturesLeaked or false alarmsNear-twinsOutcomesOther scanners, same cell
Must not flag · Tool-corroborated18False alarmsat most 17.6%0 of 18 controls flagged18 quiet · 0 flagged
  • flare-redact at most 17.6%
  • gitleaks at most 32.8%
  • trufflehog at most 17.6%
Must not flag · Project policy8False alarmsat most 32.4%0 of 8 controls flagged8 quiet · 0 flagged
  • flare-redact at most 47.1%
  • gitleaks at most 32.4%
  • trufflehog at most 32.4%
Must redact · Provider-documented19Secret spans left readableat most 16.8%0 of 19 spansNear-twins told apartat least 72.2%10 of 10 pairs19 redacted · 0 too much · 0 partly exposed · 0 missed
  • flare-redact at most 99.1%
  • gitleaks at most 16.8%
  • trufflehog at most 100.0%
45 of 45 rows
Fixtures
45
Left readable
0
Redacted too much
0
False alarms
0

Fixtures for this detector

45 fixtures. Choose "Every scanner" to see each scanner's outcome for the same rows.
Fixtures in ClickHouse Cloud API secrets
FixtureKind and evidenceredact-secret
clickhouse-cloud-api-secret-actions-envclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-bare-proseclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-bearer-headerclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-chat-pasteclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-compose-envclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-curl-user-literalclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-curl-user-varclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-dotenvclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-dotenv-key-id-pairclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-exportclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-json-api-keyclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-json-tokenclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-key-guidance-proseclickhouse-cloud · prose-mentionMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-key-shape-bareclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-key-shape-quotedclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-key-shape-unicode-crlfclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-python-requests-authclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-sdk-kwargclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-terraform-token-secretclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-trailing-hyphen-twinclickhouse-cloud · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-trailing-underscore-twinclickhouse-cloud · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-x-api-key-headerclickhouse-cloud · documented-format-literalMust redactT1 · Provider-documentedRedacted
clickhouse-cloud-api-secret-actions-secret-referenceclickhouse-cloud · templated-referenceMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-base64-run-mid-encoded-valueclickhouse-cloud · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-body-37-twinclickhouse-cloud · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-body-39-twinclickhouse-cloud · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-docs-placeholders-placeholderclickhouse-cloud · documentation-placeholderMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-ellipsis-placeholderclickhouse-cloud · documentation-placeholderMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-env-reference-referenceclickhouse-cloud · templated-referenceMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-hyphen-in-body-twinclickhouse-cloud · wrong-alphabetMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-key-id-alone-public-idclickhouse-cloud · public-identifierMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-knowledge-base-39-byte-total-twinclickhouse-cloud · wrong-lengthMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-label-proseclickhouse-cloud · benign-lookalikeMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-leading-glue-twinclickhouse-cloud · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-leading-underscore-twinclickhouse-cloud · boundary-violationMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-maskclickhouse-cloud · benign-lookalikeMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-prefix-onlyclickhouse-cloud · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-referenceclickhouse-cloud · benign-lookalikeMust not flagT3 · Project policyQuiet
clickhouse-cloud-api-secret-sha1-digest-4b1d-encoded-valueclickhouse-cloud · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-sha256-digest-4b1d-encoded-valueclickhouse-cloud · benign-encoded-valueMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-short-bodyclickhouse-cloud · benign-lookalikeMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-truncated-near-missclickhouse-cloud · format-near-missMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-uppercase-prefix-twinclickhouse-cloud · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet
clickhouse-cloud-api-secret-uuid-with-4b1d-public-idclickhouse-cloud · public-identifierMust not flagT2 · Tool-corroboratedQuiet
clickhouse-cloud-api-secret-wrong-prefix-4b1c-twinclickhouse-cloud · prefix-near-missMust not flagT2 · Tool-corroborated · twinQuiet