Comparison
Scanners and where they ran
The scanners this benchmark ran with: the version of each, how it was installed, how it was run, where it was observed and what was left out. Results are on the report and comparison pages.
4 scanners
In the order the run lists them. The kind and the description come from the scanner registry, the version from the run.
| Scanner | Kind | Version | Pinned in | Mode line of this run |
|---|---|---|---|---|
| flare-redact | Runtime library | 1.6.1 | package.json | Published npm package · secrets-only (pii, generic_assignment disabled) · JavaScript engine |
| gitleaks | Repository scanner | 8.30.1 | qualification/suite-v1.json | Directory scan · default rules |
| redact-secret | Runtime library | 0.1.0-beta.14 | qualification/suite-v1.json | Published npm package · default detectors |
| trufflehog | Repository scanner | 3.97.4 | qualification/suite-v1.json | Filesystem scan · verification disabled |
Published and candidate
Published This run measured the released redact-secret 0.1.0-beta.14.
Published measures the released npm package. Candidate measures an unreleased redact-secret build at a named commit. The other scanners run at their pinned release in both modes, so a stable count always names which of the two it came from.
Runtime library
flare-redact 1.6.1
Built to redact secrets and personal data from text at runtime. Run secrets-only here: its personal-data and generic-assignment detectors are off.
Install and pin
- Pinned version
1.6.1package.json- Observed in this run
1.6.1the version the scanner reported- Installed from
- npm, flare-redactpackage-lock.json holds 1.6.1 with integrity sha512-13Htu6VPk2t…
How it ran
- Mode line
- Published npm package · secrets-only (pii, generic_assignment disabled) · JavaScript engine
- Configuration
- Not recorded
- Build
- releasedas the official run stamped it
- Configuration hash
sha256:f111f…recorded by the official run, the same in every population it ran- Runtime comparison call
redact(), synchronous
Where it ran
- Observed
- Official run, public-evidence-snapshotrun sha256:4bec6… · credential-eval 0.1.0-alpha.16 · evidence snapshot-2026.10.06.4 · recorded 2026-10-07. 7,036 cases of the public-evidence-snapshot population; the other populations were run separately and are not added in.
- Measured
- 2026-10-07 13:39 UTCthe engine's start time in the run's artifact, finished 2026-10-07 13:39 UTC; non-semantic, outside the run's identity
- Engine host
linux-x86_64the OS and architecture the engine stamped into the artifact- OS release, CPU, Node and CI image
- Unavailablenot in this run's record (recorded 2026-10-07): it was recorded before the run driver captured host facts, and a recorded run is never amended. The engine's RunArtifact (v1) records only the OS and architecture; the OS release, CPU, Node and CI image are recorded by the benchmark's run driver when it starts the engine.
- Runtime comparison
- linux arm64 · Node v22.22.2Apple M4 (Docker Desktop linux/arm64 VM), 4 CPUs · flare-redact 1.6.1 (published npm package) · 2026-10-07
Rules
- Rules
- 81rule file 1.6.1, spec/detectors.json · 38 are mapped to a taxonomy family
Out of scope
- Personal-data detectors and generic_assignment: disabled for this run.
- The Python and Rust engines: not published to a registry, so not measured; the JavaScript engine only.
- Confidence tuning: package defaults.
Repository scanner
gitleaks 8.30.1
Built to find secrets in git history, files and directories before they are committed.
Install and pin
- Pinned version
8.30.1qualification/suite-v1.json- Observed in this run
8.30.1the version the scanner reported- Installed from
- Release archivegitleaks/gitleaks v8.30.1. The SHA-256 of each platform's archive is pinned in scanners/peer-checksums.json and checked before anything is extracted or run.
- Archive, linux-x64
gitleaks_8.30.1_linux_x64.tar.gzSHA-256 551f6fc83ea4…, pinned- Archive, linux-arm64
gitleaks_8.30.1_linux_arm64.tar.gzSHA-256 e4a487ee7ccd…, pinned- Archive, darwin-x64
gitleaks_8.30.1_darwin_x64.tar.gzSHA-256 dfe101a4db22…, pinned- Archive, darwin-arm64
gitleaks_8.30.1_darwin_arm64.tar.gzSHA-256 b40ab0ae55c5…, pinned- Location
- Read-only directory first on PATHnpm run peers:provision makes the directory read-only, so the scanner cannot update itself.
How it ran
- Mode line
- Directory scan · default rules
- Configuration
- Not recorded
- Build
- releasedas the official run stamped it
- Configuration hash
sha256:f091a…recorded by the official run, the same in every population it ran
Where it ran
- Observed
- Official run, public-evidence-snapshotrun sha256:4bec6… · credential-eval 0.1.0-alpha.16 · evidence snapshot-2026.10.06.4 · recorded 2026-10-07. 7,036 cases of the public-evidence-snapshot population; the other populations were run separately and are not added in.
- Measured
- 2026-10-07 13:39 UTCthe engine's start time in the run's artifact, finished 2026-10-07 13:39 UTC; non-semantic, outside the run's identity
- Engine host
linux-x86_64the OS and architecture the engine stamped into the artifact- OS release, CPU, Node and CI image
- Unavailablenot in this run's record (recorded 2026-10-07): it was recorded before the run driver captured host facts, and a recorded run is never amended. The engine's RunArtifact (v1) records only the OS and architecture; the OS release, CPU, Node and CI image are recorded by the benchmark's run driver when it starts the engine.
Rules
- Rules
- 222rule file 8.30.1, config/gitleaks.toml · 73 are mapped to a taxonomy family
Out of scope
- Git history scanning: only the directory scan of the fixture files is run.
- Custom rule files and environment rule overrides: default rules only.
Runtime library
redact-secret 0.1.0-beta.14
Install and pin
- Pinned version
0.1.0-beta.14qualification/suite-v1.json- Observed in this run
0.1.0-beta.14the version the scanner reported- Installed from
- npm, @redact-secret/corepackage-lock.json holds 0.1.0-beta.14 with integrity sha512-1h5NxUto2ZE…
How it ran
- Mode line
- Published npm package · default detectors
- Adapter
- adapter v3 · family mapping v2
- Detectors enabled
- default
- Runs on
- node
- Build
- releasedas the official run stamped it
- Configuration hash
sha256:e8d78…recorded by the official run, the same in every population it ran- Runtime comparison call
scanAndRedact(), synchronous
Where it ran
- Observed
- Official run, public-evidence-snapshotrun sha256:4bec6… · credential-eval 0.1.0-alpha.16 · evidence snapshot-2026.10.06.4 · recorded 2026-10-07. 7,036 cases of the public-evidence-snapshot population; the other populations were run separately and are not added in.
- Measured
- 2026-10-07 13:39 UTCthe engine's start time in the run's artifact, finished 2026-10-07 13:39 UTC; non-semantic, outside the run's identity
- Engine host
linux-x86_64the OS and architecture the engine stamped into the artifact- OS release, CPU, Node and CI image
- Unavailablenot in this run's record (recorded 2026-10-07): it was recorded before the run driver captured host facts, and a recorded run is never amended. The engine's RunArtifact (v1) records only the OS and architecture; the OS release, CPU, Node and CI image are recorded by the benchmark's run driver when it starts the engine.
- Runtime comparison
- linux arm64 · Node v22.22.2Apple M4 (Docker Desktop linux/arm64 VM), 4 CPUs · redact-secret 0.1.0-beta.14 (published npm package) · 2026-10-07
Rules
- Registered detectors
- 120benchmarks/detectors.json, read at redact-secret 5fddf1a60d02; this run measured 0.1.0-beta.14, so the count is of another revision
Out of scope
- Statements read at
redact-secret 422e43e3dc1fthe product commit whose decision records the statements restate- Bound to
- 0.1.0-beta.14 · Published npm package · default detectorscommit 0c62fd38bca7. readAt.revision is an ancestor of the v0.1.0-beta.14 tag (0c62fd38), and each product decision record cited in sources has the same git blob at both commits (checked 2026-10-07), so the statements read at readAt are unchanged at this release.
- Measured
- 0.1.0-beta.14 · Published npm package · default detectorsas the official run sha256:4bec6… recorded it, configuration sha256:e8d78…
- Binding
- Currentthe measured release and mode line are the ones the statements are bound to
Credential scope the product documents
Restated from the product's own decision records: what it does not detect or read, in any run.
- Credentials cut by a line break, string-literal operator, continuation, markdown line ending or escaped newline: not reconstructed.
- Base64, hex, percent-encoded and other encoded carriers: not decoded; a raw encoded value claims nothing.
- No declared netrc, kubeconfig user token, HTTP session cookie, Azure SAS, S3-presigned or GCS-signed-URL family.
- Values in those carriers are claimed only when a supported detector reads them independently, and that claim is incidental.
- JSON {"name": ..., "value": ...} object pairs (HAR postData params and queryString members): not read as a name with its value.
- A credential name that keys an object holding a "value" member (Terraform state outputs): not read; the one-line assignment form is.
- A token behind a percent-encoded delimiter (%22, %3D, %2F): not read for any family; percent-encoding is not decoded.
- HTTP session cookies in HAR headers and cookies arrays: no session-cookie family, and no name/value pair reading.
Optional personal-data detection
A product capability that is off in the measured configuration. Not measured by this run, which is not a statement that the product lacks it.
- The personal-data profile, selectors and the pii:us add-on: off in the published runs; default credential detectors only.
Surfaces this benchmark does not run
Not measured here, which is not a statement about what the product supports.
- The CLI, WebAssembly and Python surfaces: not run; the Node package's public API only.
Repository scanner
trufflehog 3.97.4
Built to find and verify secrets in repositories and other sources. Verification is off here.
Install and pin
- Pinned version
3.97.4qualification/suite-v1.json- Observed in this run
3.97.4the version the scanner reported- Installed from
- Release archivetrufflesecurity/trufflehog v3.97.4. The SHA-256 of each platform's archive is pinned in scanners/peer-checksums.json and checked before anything is extracted or run.
- Archive, linux-x64
trufflehog_3.97.4_linux_amd64.tar.gzSHA-256 dc24007c2f23…, pinned- Archive, linux-arm64
trufflehog_3.97.4_linux_arm64.tar.gzSHA-256 7e65e771d2a2…, pinned- Archive, darwin-x64
trufflehog_3.97.4_darwin_amd64.tar.gzSHA-256 36799557198a…, pinned- Archive, darwin-arm64
trufflehog_3.97.4_darwin_arm64.tar.gzSHA-256 57e2a41c1e19…, pinned- Location
- Read-only directory first on PATHnpm run peers:provision makes the directory read-only, so the scanner cannot update itself.
How it ran
- Mode line
- Filesystem scan · verification disabled
- Configuration
- Not recorded
- Build
- releasedas the official run stamped it
- Configuration hash
sha256:d077d…recorded by the official run, the same in every population it ran
Where it ran
- Observed
- Official run, public-evidence-snapshotrun sha256:4bec6… · credential-eval 0.1.0-alpha.16 · evidence snapshot-2026.10.06.4 · recorded 2026-10-07. 7,036 cases of the public-evidence-snapshot population; the other populations were run separately and are not added in.
- Measured
- 2026-10-07 13:39 UTCthe engine's start time in the run's artifact, finished 2026-10-07 13:39 UTC; non-semantic, outside the run's identity
- Engine host
linux-x86_64the OS and architecture the engine stamped into the artifact- OS release, CPU, Node and CI image
- Unavailablenot in this run's record (recorded 2026-10-07): it was recorded before the run driver captured host facts, and a recorded run is never amended. The engine's RunArtifact (v1) records only the OS and architecture; the OS release, CPU, Node and CI image are recorded by the benchmark's run driver when it starts the engine.
Rules
- Rules
- 892rule file 3.97.4, pkg/engine/defaults/defaults.go · 83 are mapped to a taxonomy family
Out of scope
- Live verification of findings: off, so no network calls are made.
- Sources other than a filesystem scan of the fixture files.