# Evaluation

> Measured results for Nightshade 0.1.1: showcase examples, the synthetic corpus and real contracts, with denominators, misses, versions and how to reproduce.

Canonical: https://nightshade.dicksonwu.xyz/evaluation.html
Release: Nightshade 0.1.1 (v0.1.1)

This page collects the numbers I publish about Nightshade, with the denominator, version and limits next to each one. All of them come from one proof run of the released 0.1.1 CLI, installed from the release tarball on 2026-10-24.

- **10 / 10 showcase findings fire.** 10 corrected versions and 9 lookalikes stay quiet. Synthetic, one rule per scan, compiler-validated. https://nightshade.dicksonwu.xyz/examples.md
- **263 / 266 synthetic positive checks detected.** 3 missed, all for one rule. Controls quiet: 240/242 corrected, 237/252 lookalike. 7 incomplete. 93 rules. https://nightshade.dicksonwu.xyz/evaluation.md#synthetic-corpus
- **258 / 300 real contract roots fully analyzed.** Baseline analysis. 237/300 complete for every selected rule; 27 compiler-rejected roots stay in the denominator; 0 process failures. Aggregate only. https://nightshade.dicksonwu.xyz/evaluation.md#real-contracts

Nightshade 0.1.1, proof run on 2026-10-24. None of these numbers is a precision or recall estimate.

## Showcase examples

Each of the 10 [examples](https://nightshade.dicksonwu.xyz/examples.md) selects one rule. The positive fired in 10 of 10, the corrected version was quiet in 10 of 10 and the lookalike was quiet in 9 of 9 (one example has no lookalike). Every variant passed Compact 0.34.0 compiler validation and complete analysis. Exit codes are from the same runs.

| Example | Rule | Severity | Positive | Corrected | Lookalike |
| --- | --- | --- | --- | --- | --- |
| [Private authentication preimage reaches the public ledger](https://nightshade.dicksonwu.xyz/examples.md#ledger-bound-preimage-publication) | `ledger-bound-preimage-publication` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [A public key comparison does not prove ownership](https://nightshade.dicksonwu.xyz/examples.md#spoofable-authorization) | `spoofable-authorization` | high | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [The signature verification result is discarded](https://nightshade.dicksonwu.xyz/examples.md#ignored-validation-result) | `ignored-validation-result` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [Caller time needs a chain-time bound](https://nightshade.dicksonwu.xyz/examples.md#freshness-bound-caller-time) | `freshness-bound-caller-time` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [Initialization can run repeatedly](https://nightshade.dicksonwu.xyz/examples.md#re-initialization) | `re-initialization` | low | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [Two shielded mints reuse the same descriptor](https://nightshade.dicksonwu.xyz/examples.md#duplicate-shielded-mint) | `duplicate-shielded-mint` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [The Merkle membership result is ignored](https://nightshade.dicksonwu.xyz/examples.md#merkle-membership-not-enforced) | `merkle-membership-not-enforced` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [An unchecked witness value becomes public state](https://nightshade.dicksonwu.xyz/examples.md#unconstrained-witness-to-state) | `unconstrained-witness-to-state` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [A freshness check never consumes the nullifier](https://nightshade.dicksonwu.xyz/examples.md#nullifier-check-without-insert) | `nullifier-check-without-insert` | medium | 1 finding, exit 1 | quiet, exit 0 | quiet, exit 0 |
| [A private commitment uses fixed randomness](https://nightshade.dicksonwu.xyz/examples.md#predictable-commitment-randomness) | `predictable-commitment-randomness` | medium | 1 finding, exit 1 | quiet, exit 0 | none |

## Synthetic corpus

The repository's test corpus pairs each rule with positive cases that should match and corrected, lookalike or other negative cases that should stay quiet. I ran every case declared in the five corpus registries with the installed CLI.

The run made 915 distinct CLI scans covering 1161 declarations across 93 rules. 915 scans passed compiler validation, 7 were incomplete and 0 had process failures. A check counts only when the compiler accepted the file and analysis completed. Registries overlap, so the last row counts each (scan, rule, role) once.

| Scope | Positive detected | Corrected quiet | Lookalike quiet | Other negative quiet | Incomplete |
| --- | ---: | ---: | ---: | ---: | ---: |
| `detector-counterexamples/manifest.json` | 20/20 | 20/20 | 20/20 | 0/0 | 0 |
| `known-bugs.json` | 105/105 | 105/105 | 105/105 | 0/0 | 0 |
| `manifest.json` | 130/130 | 106/106 | 109/121 | 89/96 | 7 |
| `policies-manifest.json` | 6/6 | 6/6 | 0/0 | 0/0 | 0 |
| `synthetic-regressions.json` | 104/107 | 105/107 | 104/107 | 0/0 | 0 |
| **All declarations (overlap kept)** | 365/368 | 342/344 | 338/353 | 89/96 | 7 |
| **Unique checks** | 263/266 | 240/242 | 237/252 | 89/96 | 7 |

**Misses and control hits.** 3 positive checks were missed, all for [`identity-signature-key`](https://nightshade.dicksonwu.xyz/rules/identity-signature-key.md). 17 corrected or lookalike controls still matched; the corpus labels all of them as expected matches (0 were unexpected), and they get no quiet credit. 7 checks were incomplete and count as failures, not as quiet. 21 synthetic triplets have no mapped target rule; they were scanned but not scored.

Corpus scans used budgets of 100,000 steps, 256 paths and 100,000 abstract values, with no suppressions, severity filters or baselines. The run took 1105 seconds on 4 CPU cores.

### Results for each rule

| Rule | Positive detected | Corrected quiet | Lookalike quiet | Other negative quiet | Incomplete |
| --- | ---: | ---: | ---: | ---: | ---: |
| [`always-true-assertion`](https://nightshade.dicksonwu.xyz/rules/always-true-assertion.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`caller-key-zero-entry-review`](https://nightshade.dicksonwu.xyz/rules/caller-key-zero-entry-review.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`caller-supplied-deadline`](https://nightshade.dicksonwu.xyz/rules/caller-supplied-deadline.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`caller-supplied-ledger-equality`](https://nightshade.dicksonwu.xyz/rules/caller-supplied-ledger-equality.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`checked-digest-narrowing`](https://nightshade.dicksonwu.xyz/rules/checked-digest-narrowing.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`checkpoint-splits-atomic-update`](https://nightshade.dicksonwu.xyz/rules/checkpoint-splits-atomic-update.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`circuit-cost`](https://nightshade.dicksonwu.xyz/rules/circuit-cost.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`circuit-work-budget`](https://nightshade.dicksonwu.xyz/rules/circuit-work-budget.md) | 2/2 | 2/2 | 1/1 | 0/0 | 0 |
| [`closed-membership-capacity-review`](https://nightshade.dicksonwu.xyz/rules/closed-membership-capacity-review.md) | 6/6 | 3/3 | 2/2 | 10/10 | 0 |
| [`conditional-hash-range`](https://nightshade.dicksonwu.xyz/rules/conditional-hash-range.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`configured-membership-capacity-review`](https://nightshade.dicksonwu.xyz/rules/configured-membership-capacity-review.md) | 3/3 | 3/3 | 3/3 | 0/0 | 0 |
| [`constant-condition`](https://nightshade.dicksonwu.xyz/rules/constant-condition.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`current-root-policy-mismatch`](https://nightshade.dicksonwu.xyz/rules/current-root-policy-mismatch.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`dead-code`](https://nightshade.dicksonwu.xyz/rules/dead-code.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`declared-secret-publication`](https://nightshade.dicksonwu.xyz/rules/declared-secret-publication.md) | 3/3 | 3/3 | 2/2 | 0/0 | 0 |
| [`default-recipient-or-owner`](https://nightshade.dicksonwu.xyz/rules/default-recipient-or-owner.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`destructive-list-reset`](https://nightshade.dicksonwu.xyz/rules/destructive-list-reset.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`discarded-comparison`](https://nightshade.dicksonwu.xyz/rules/discarded-comparison.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`discarded-shielded-send-result`](https://nightshade.dicksonwu.xyz/rules/discarded-shielded-send-result.md) | 5/5 | 5/5 | 5/5 | 0/0 | 0 |
| [`duplicate-shielded-mint`](https://nightshade.dicksonwu.xyz/rules/duplicate-shielded-mint.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`duplicated-branch-tail`](https://nightshade.dicksonwu.xyz/rules/duplicated-branch-tail.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`either-payload-mismatch`](https://nightshade.dicksonwu.xyz/rules/either-payload-mismatch.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`enrolled-signature-key-review`](https://nightshade.dicksonwu.xyz/rules/enrolled-signature-key-review.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`exact-balance-threshold`](https://nightshade.dicksonwu.xyz/rules/exact-balance-threshold.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`exported-shielded-descriptor`](https://nightshade.dicksonwu.xyz/rules/exported-shielded-descriptor.md) | 3/3 | 3/3 | 3/3 | 0/0 | 0 |
| [`field-arithmetic-wraparound`](https://nightshade.dicksonwu.xyz/rules/field-arithmetic-wraparound.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`field-recomposition-range-review`](https://nightshade.dicksonwu.xyz/rules/field-recomposition-range-review.md) | 5/5 | 4/5 | 4/5 | 0/0 | 0 |
| [`forbidden-state-transition`](https://nightshade.dicksonwu.xyz/rules/forbidden-state-transition.md) | 2/2 | 2/2 | 1/1 | 0/0 | 0 |
| [`freshness-bound-caller-time`](https://nightshade.dicksonwu.xyz/rules/freshness-bound-caller-time.md) | 12/12 | 12/12 | 12/12 | 0/0 | 0 |
| [`generic-witness-maybe-abi`](https://nightshade.dicksonwu.xyz/rules/generic-witness-maybe-abi.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`identity-signature-key`](https://nightshade.dicksonwu.xyz/rules/identity-signature-key.md) | 1/4 | 4/4 | 4/4 | 0/0 | 0 |
| [`ignored-validation-result`](https://nightshade.dicksonwu.xyz/rules/ignored-validation-result.md) | 3/3 | 3/3 | 3/3 | 0/0 | 0 |
| [`impossible-assertion`](https://nightshade.dicksonwu.xyz/rules/impossible-assertion.md) | 2/2 | 2/2 | 1/1 | 0/0 | 0 |
| [`inconsistent-access-control`](https://nightshade.dicksonwu.xyz/rules/inconsistent-access-control.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`inconsistent-keyed-state-guard`](https://nightshade.dicksonwu.xyz/rules/inconsistent-keyed-state-guard.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`ledger-bound-preimage-publication`](https://nightshade.dicksonwu.xyz/rules/ledger-bound-preimage-publication.md) | 7/7 | 12/12 | 1/1 | 0/0 | 0 |
| [`low-entropy-hash-disclosure`](https://nightshade.dicksonwu.xyz/rules/low-entropy-hash-disclosure.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`membership-identity-binding-review`](https://nightshade.dicksonwu.xyz/rules/membership-identity-binding-review.md) | 4/4 | 3/3 | 2/3 | 5/5 | 0 |
| [`merkle-insert-without-dedupe`](https://nightshade.dicksonwu.xyz/rules/merkle-insert-without-dedupe.md) | 2/2 | 2/2 | 1/2 | 0/0 | 0 |
| [`merkle-leaf-binding-review`](https://nightshade.dicksonwu.xyz/rules/merkle-leaf-binding-review.md) | 6/6 | 6/6 | 6/6 | 0/0 | 0 |
| [`merkle-membership-not-enforced`](https://nightshade.dicksonwu.xyz/rules/merkle-membership-not-enforced.md) | 5/5 | 5/5 | 5/5 | 0/0 | 0 |
| [`missing-role-bootstrap`](https://nightshade.dicksonwu.xyz/rules/missing-role-bootstrap.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`mutually-exclusive-terminal-payout`](https://nightshade.dicksonwu.xyz/rules/mutually-exclusive-terminal-payout.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`native-key-authentication-review`](https://nightshade.dicksonwu.xyz/rules/native-key-authentication-review.md) | 5/5 | 5/5 | 5/5 | 0/0 | 0 |
| [`nullifier-check-without-insert`](https://nightshade.dicksonwu.xyz/rules/nullifier-check-without-insert.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`openzeppelin-preconditions`](https://nightshade.dicksonwu.xyz/rules/openzeppelin-preconditions.md) | 6/6 | 6/6 | 6/6 | 0/0 | 0 |
| [`operation-intent-policy`](https://nightshade.dicksonwu.xyz/rules/operation-intent-policy.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`permissionless-mint-egress`](https://nightshade.dicksonwu.xyz/rules/permissionless-mint-egress.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`predictable-commitment-randomness`](https://nightshade.dicksonwu.xyz/rules/predictable-commitment-randomness.md) | 2/2 | 2/2 | 1/1 | 0/0 | 0 |
| [`public-key-membership-authorization`](https://nightshade.dicksonwu.xyz/rules/public-key-membership-authorization.md) | 9/9 | 6/6 | 6/6 | 3/3 | 0 |
| [`public-two-component-domain-review`](https://nightshade.dicksonwu.xyz/rules/public-two-component-domain-review.md) | 6/6 | 5/5 | 3/4 | 4/5 | 1 |
| [`publicly-derivable-encryption-secret`](https://nightshade.dicksonwu.xyz/rules/publicly-derivable-encryption-secret.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`publicly-derived-guard-tag`](https://nightshade.dicksonwu.xyz/rules/publicly-derived-guard-tag.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`published-digest-generator-scalar-review`](https://nightshade.dicksonwu.xyz/rules/published-digest-generator-scalar-review.md) | 4/4 | 4/4 | 4/4 | 6/6 | 0 |
| [`quorum-signer-uniqueness-review`](https://nightshade.dicksonwu.xyz/rules/quorum-signer-uniqueness-review.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`re-initialization`](https://nightshade.dicksonwu.xyz/rules/re-initialization.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`received-coin-not-validated`](https://nightshade.dicksonwu.xyz/rules/received-coin-not-validated.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`redundant-boolean-branch`](https://nightshade.dicksonwu.xyz/rules/redundant-boolean-branch.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`redundant-self-shielded-receive`](https://nightshade.dicksonwu.xyz/rules/redundant-self-shielded-receive.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`resettable-list-nonce`](https://nightshade.dicksonwu.xyz/rules/resettable-list-nonce.md) | 4/4 | 4/4 | 4/4 | 0/0 | 0 |
| [`resettable-memo-nonce-review`](https://nightshade.dicksonwu.xyz/rules/resettable-memo-nonce-review.md) | 2/2 | 2/2 | 7/7 | 0/0 | 0 |
| [`secret-binding-divergence-review`](https://nightshade.dicksonwu.xyz/rules/secret-binding-divergence-review.md) | 7/7 | 2/2 | 2/3 | 9/9 | 0 |
| [`shadowing`](https://nightshade.dicksonwu.xyz/rules/shadowing.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`shared-module-once-gate`](https://nightshade.dicksonwu.xyz/rules/shared-module-once-gate.md) | 2/2 | 2/2 | 2/2 | 1/1 | 0 |
| [`shielded-burn-amount-width`](https://nightshade.dicksonwu.xyz/rules/shielded-burn-amount-width.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`shielded-coin-runtime-width`](https://nightshade.dicksonwu.xyz/rules/shielded-coin-runtime-width.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`shielded-contract-recipient-review`](https://nightshade.dicksonwu.xyz/rules/shielded-contract-recipient-review.md) | 2/2 | 2/2 | 2/2 | 1/1 | 0 |
| [`signature-key-identity-exclusion-review`](https://nightshade.dicksonwu.xyz/rules/signature-key-identity-exclusion-review.md) | 3/3 | 3/3 | 2/5 | 1/1 | 0 |
| [`signed-effect-input`](https://nightshade.dicksonwu.xyz/rules/signed-effect-input.md) | 3/3 | 3/3 | 3/3 | 0/0 | 0 |
| [`source-claim-map-getter-review`](https://nightshade.dicksonwu.xyz/rules/source-claim-map-getter-review.md) | 4/4 | 2/2 | 2/2 | 10/10 | 0 |
| [`source-claim-shielded-issuance-review`](https://nightshade.dicksonwu.xyz/rules/source-claim-shielded-issuance-review.md) | 6/6 | 2/2 | 15/16 | 0/1 | 1 |
| [`split-witness-identity`](https://nightshade.dicksonwu.xyz/rules/split-witness-identity.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`spoofable-authorization`](https://nightshade.dicksonwu.xyz/rules/spoofable-authorization.md) | 28/28 | 28/28 | 28/28 | 0/0 | 0 |
| [`stale-nft-metadata`](https://nightshade.dicksonwu.xyz/rules/stale-nft-metadata.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`stored-schnorr-key-enrollment-review`](https://nightshade.dicksonwu.xyz/rules/stored-schnorr-key-enrollment-review.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`time-dependence`](https://nightshade.dicksonwu.xyz/rules/time-dependence.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`transient-digest-to-ledger`](https://nightshade.dicksonwu.xyz/rules/transient-digest-to-ledger.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`unbounded-mint`](https://nightshade.dicksonwu.xyz/rules/unbounded-mint.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`unconstrained-witness-to-state`](https://nightshade.dicksonwu.xyz/rules/unconstrained-witness-to-state.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`unguarded-sensitive-state-write`](https://nightshade.dicksonwu.xyz/rules/unguarded-sensitive-state-write.md) | 3/3 | 3/3 | 2/2 | 0/0 | 0 |
| [`unnecessary-disclose`](https://nightshade.dicksonwu.xyz/rules/unnecessary-disclose.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`unprotected-value-transfer`](https://nightshade.dicksonwu.xyz/rules/unprotected-value-transfer.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`unshielded-refund-endpoint-review`](https://nightshade.dicksonwu.xyz/rules/unshielded-refund-endpoint-review.md) | 5/5 | 1/1 | 1/3 | 7/9 | 2 |
| [`unsupported-contract-owner`](https://nightshade.dicksonwu.xyz/rules/unsupported-contract-owner.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`unused-declaration`](https://nightshade.dicksonwu.xyz/rules/unused-declaration.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`write-only-ledger`](https://nightshade.dicksonwu.xyz/rules/write-only-ledger.md) | 2/2 | 1/2 | 1/2 | 0/0 | 0 |
| [`zero-amount-complementary-send-review`](https://nightshade.dicksonwu.xyz/rules/zero-amount-complementary-send-review.md) | 4/4 | 2/2 | 3/4 | 7/8 | 1 |
| [`zero-amount-initial-membership`](https://nightshade.dicksonwu.xyz/rules/zero-amount-initial-membership.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`zero-amount-set-consumption-review`](https://nightshade.dicksonwu.xyz/rules/zero-amount-set-consumption-review.md) | 2/2 | 2/2 | 4/5 | 8/8 | 0 |
| [`zero-amount-unguarded-map-debit-review`](https://nightshade.dicksonwu.xyz/rules/zero-amount-unguarded-map-debit-review.md) | 2/2 | 2/2 | 2/2 | 0/0 | 0 |
| [`zero-amount-unshielded-send`](https://nightshade.dicksonwu.xyz/rules/zero-amount-unshielded-send.md) | 1/1 | 1/1 | 1/1 | 0/0 | 0 |
| [`zero-default-getter-membership-review`](https://nightshade.dicksonwu.xyz/rules/zero-default-getter-membership-review.md) | 7/7 | 4/4 | 2/2 | 14/14 | 0 |
| [`zero-transfer-retained-list-review`](https://nightshade.dicksonwu.xyz/rules/zero-transfer-retained-list-review.md) | 3/3 | 2/2 | 3/4 | 3/5 | 2 |

### All 31 deviations

Every failed check, exact-line disagreement and incomplete observation. Rows are manifest declarations, so two rows can describe the same scan.

| Registry / case | Rule | Kind | Expected to observed lines | Issue |
| --- | --- | --- | --- | --- |
| known-bugs.json / authorization-raw-native-witness | `spoofable-authorization` | positive | [7] to [6] | exact expectation mismatch |
| synthetic-regressions.json / merkle-insert-without-dedupe-39 | `merkle-insert-without-dedupe` | lookalike | [11] to [11] | control hit |
| synthetic-regressions.json / write-only-ledger-70 | `write-only-ledger` | corrected | [3] to [3] | control hit |
| synthetic-regressions.json / write-only-ledger-70 | `write-only-ledger` | lookalike | [3] to [3] | control hit |
| synthetic-regressions.json / identity-signature-key-71 | `identity-signature-key` | positive | [6] to [] | miss, exact expectation mismatch |
| synthetic-regressions.json / identity-signature-key-71 | `identity-signature-key` | lookalike | [7] to [] | exact expectation mismatch |
| synthetic-regressions.json / field-recomposition-range-review-86 | `field-recomposition-range-review` | corrected | [7] to [7] | control hit |
| synthetic-regressions.json / field-recomposition-range-review-86 | `field-recomposition-range-review` | lookalike | [7] to [7] | control hit |
| synthetic-regressions.json / identity-signature-key-96 | `identity-signature-key` | positive | [4,7] to [] | miss, exact expectation mismatch |
| synthetic-regressions.json / identity-signature-key-96 | `identity-signature-key` | lookalike | [5,9] to [] | exact expectation mismatch |
| synthetic-regressions.json / identity-signature-key-97 | `identity-signature-key` | positive | [10,13] to [] | miss, exact expectation mismatch |
| synthetic-regressions.json / identity-signature-key-97 | `identity-signature-key` | lookalike | [11,15] to [] | exact expectation mismatch |
| manifest.json / zeroAmountSetConsumptionReview40/drafts/consumption40/intentional-zero-consumption.compact | `zero-amount-set-consumption-review` | lookalike | [15] to [15] | control hit |
| manifest.json / publicTwoComponentDomainReview43/drafts/owner-domain43/lookalike.compact | `public-two-component-domain-review` | lookalike | [15] to [15] | control hit |
| manifest.json / publicTwoComponentDomainReview43/drafts/owner-domain43/opaque-salt-guard.compact | `public-two-component-domain-review` | negative | [] to [] | incomplete or unvalidated; domain-review-evidence-unavailable |
| manifest.json / signatureIdentityExclusion46/drafts/signature46/caller-key/lookalike.compact | `signature-key-identity-exclusion-review` | lookalike | [5,9] to [5,9] | control hit |
| manifest.json / signatureIdentityExclusion46/drafts/signature46/persistent-key/lookalike.compact | `signature-key-identity-exclusion-review` | lookalike | [12,17] to [12,17] | control hit |
| manifest.json / signatureIdentityExclusion46/drafts/signature46/counter/lookalike.compact | `signature-key-identity-exclusion-review` | lookalike | [44,45] to [44,45] | control hit |
| manifest.json / complementarySendReview47/drafts/complementary47/boundaries/intentional-same-executable-as-positive.compact | `zero-amount-complementary-send-review` | lookalike | [9] to [9] | control hit |
| manifest.json / complementarySendReview47/drafts/complementary47/boundaries/persistent-map-lost-lineage-refusal.compact | `zero-amount-complementary-send-review` | negative | [] to [] | incomplete or unvalidated; complementary-send-proof-unavailable, complementary-send-preparation-unavailable |
| manifest.json / secretBindingDivergence49/drafts/secret-binding49/controls/intentional-weaker-same-key.compact | `secret-binding-divergence-review` | lookalike | [18] to [18] | control hit |
| manifest.json / zeroListGrowth50/drafts/zero-list50/controls/intentional-zero-linked-growth.compact | `zero-transfer-retained-list-review` | lookalike | [12] to [12] | control hit |
| manifest.json / zeroListGrowth50/drafts/zero-list50/controls/append-then-reset.compact | `zero-transfer-retained-list-review` | negative | [] to [] | incomplete or unvalidated; list-growth-evidence-incomplete, list-growth-preparation-unavailable |
| manifest.json / zeroListGrowth50/drafts/zero-list50/controls/append-then-pop.compact | `zero-transfer-retained-list-review` | negative | [] to [] | incomplete or unvalidated; list-growth-evidence-incomplete, list-growth-preparation-unavailable |
| manifest.json / sourceClaims52/drafts/source-claims52/incomplete-unknown-tag.compact | `source-claim-shielded-issuance-review` | negative | [] to [] | incomplete or unvalidated; source-claim-evidence-unavailable, source-claim-evidence-unavailable |
| manifest.json / sourceClaims52/drafts/source-claims52/must-match-intentional-claim.compact | `source-claim-shielded-issuance-review` | lookalike | [6] to [6] | control hit |
| manifest.json / independentMembershipIdentity54/drafts/memo-identity44-boundaries/intentional-admin.compact | `membership-identity-binding-review` | lookalike | [20] to [20] | control hit |
| manifest.json / refundEndpoint57/drafts/arithmetic-endpoint57/dynamic-map-snapshots.compact | `unshielded-refund-endpoint-review` | negative | [] to [] | incomplete or unvalidated; refund-endpoint-source-unavailable |
| manifest.json / refundEndpoint57/drafts/arithmetic-endpoint57/intentional-endpoint.compact | `unshielded-refund-endpoint-review` | lookalike | [58] to [58] | control hit |
| manifest.json / refundEndpoint57/drafts/arithmetic-endpoint57/intentional.compact | `unshielded-refund-endpoint-review` | lookalike | [58] to [58] | control hit |
| manifest.json / refundEndpoint57/drafts/arithmetic-endpoint57/scalar-ledger-snapshots.compact | `unshielded-refund-endpoint-review` | negative | [] to [] | incomplete or unvalidated; refund-endpoint-source-unavailable |

## Real contracts

I reran the installed 0.1.1 CLI on a fixed set of 300 contract roots from 144 repositories. The assembled inputs are not redistributed, so only aggregate counts are published: no project names, paths or findings. All 300 roots stay in every denominator, including the 27 the compiler rejected, which were not scanned.

| Outcome | Result |
| --- | ---: |
| Contract roots | 300 (from 144 repositories) |
| Compiler accepted / rejected | 273 / 27 |
| Baseline analysis complete | 258 / 300 (86.0%) |
| All selected rules complete | 237 / 300 (79.0%) |
| Incomplete: budget limit | 12 |
| Incomplete: frontend | 1 |
| Incomplete: other | 2 |
| Process failures / worker-failure roots | 0 / 0 |
| Finding instances | 315 |
| Wall time, 4 workers | 157.9 s |
| Per root: median / p95 / max | 1.36 s / 6.26 s / 32.3 s |

The 15 accepted roots without complete baseline analysis are coverage gaps, not clean results. One root did not get the semantic profile it requested and is counted as incomplete. The 315 finding instances are static observations under the default audit selection; I did not review them for accuracy here, so they are not confirmed vulnerabilities.

Compiler accept and reject decisions came from the authenticated frozen manifest of the original run; compilers were not rerun. Source hashes for all 300 roots and 66 dependency trees were checked before and after the run and did not change. The earlier pre-release 0.1.0 measurement on the same inputs recorded the same completion counts (258/300 and 237/300) and 315 findings.

## What these numbers do not show

- There is no independently established precision or recall estimate for the release. Synthetic tests check defined rule behavior, not unknown vulnerabilities or ecosystem-wide recall.
- A quiet corrected or lookalike case shows that the selected rule stopped firing. It does not establish that a contract is secure.
- Finding counts on real contracts are static observations, not confirmed vulnerabilities, and completion counts are not accuracy.
- Incomplete analysis and compiler rejection are never counted as clean results.
- Compiler validation used `--skip-zk`, so it skips proof generation and does not establish runtime behavior.
- Results apply to version 0.1.1 and the compiler profiles shown. Other versions can differ.

## Versions and identities

| Item | Identity |
| --- | --- |
| Nightshade CLI | `Nightshade 0.1.1`, installed with npm from the release tarball |
| Release tarball SHA-256 | `fee7af6cb124d677e7b62c90da22466dd2c0b5819de9625c11de398635a0a3d6` |
| Node / npm | v24.20.0 / 11.19.0 |
| Grammar revision | `22de6ee9dbf4b92fe4e2fe3602e53f1e352b79fd` |
| Grammar build | `fef63bac6b6120f35b9e64ec29134a40a1fd5a1008ec8e8ae714f523788b5a93` |
| Compact compiler 0.31.1 binary | `3054ffa89d7a4dfe24afd31c27ef37e87a95757de0fc24485f335635e26dce57` |
| Compact compiler 0.34.0 binary | `baa50cd002492b3163ac3d25bc88904d4c7a40ad490fbeb3b970ea48c7f1d3b5` |
| Corpus source checkout | `db27d3bd3c772bd1ab181bcf0c4f86e6f8621893` (release-0.1) |
| Public import dependency | [OpenZeppelin compact-contracts](https://github.com/OpenZeppelin/compact-contracts) at `ec1331cd8195694c747d291e8fc8ab6d2914a976` |
| Proof run | 2026-10-24, CPU cores 0 to 3 |
| Public proof pack | [`nightshade-proof-0.1.1.tar.gz`](https://nightshade.dicksonwu.xyz/downloads/nightshade-proof-0.1.1.tar.gz), SHA-256 `9d95e631dc318006610882911c0d58b7c25fa7754c39b49205c309d2e2e57b3b` |

The showcase source fixtures match the files under `tests/corpus` in the repository at v0.1.1 by SHA-256; showcase files that were trimmed say so. The corpus also contains no public audit reproductions in this release, so no result here is a claim about a third-party project.

## Reproduce

[Download the proof pack](https://nightshade.dicksonwu.xyz/downloads/nightshade-proof-0.1.1.tar.gz) (`nightshade-proof-0.1.1.tar.gz`, 1.7 MB, 4692 files). SHA-256: `9d95e631dc318006610882911c0d58b7c25fa7754c39b49205c309d2e2e57b3b`

It holds the synthetic input snapshot, every raw scan output with its recorded command and exit code, the per-scan configs, these ten examples, the report and the scripts that produced and check them. Local paths in it are replaced with placeholders such as `<pack>` and `<compact-toolchains>`, the release tarball is left out (it is the v0.1.1 release asset) and real-contract records keep only the aggregate numbers on the [evaluation](https://nightshade.dicksonwu.xyz/evaluation.md) page. `PUBLIC-PACK.md` inside lists every change.

**One example.** Install the release as in [getting started](https://nightshade.dicksonwu.xyz/getting-started.md), download an example's files from the [examples page](https://nightshade.dicksonwu.xyz/examples.md), and run from that folder:

```sh
./.nightshade/node_modules/.bin/nightshade vulnerable.compact --config config.json
./.nightshade/node_modules/.bin/nightshade corrected.compact --config config.json
```

The recorded runs also passed `--compiler` with the path to a Compact 0.34.0 `compactc` to validate each file with the compiler. Expect exit code 1 and one finding for the flagged file, and exit code 0 with no findings for the corrected one.

**The corpus.** The fixtures and the five registries are in the repository under [`tests/corpus`](https://github.com/DicksonWu654/nightshade/tree/v0.1.1/tests/corpus) at v0.1.1. The [proof pack](https://nightshade.dicksonwu.xyz/downloads/nightshade-proof-0.1.1.tar.gz) holds the input snapshot, raw outputs, scripts and checksums of this run. To check it without rescanning:

```sh
tar -xzf nightshade-proof-0.1.1.tar.gz
cd nightshade-proof-0.1.1
sha256sum --check --quiet SHA256SUMS
mkdir -p release
curl -fL -o release/nightshade-compact-0.1.1.tgz https://github.com/DicksonWu654/nightshade/releases/download/v0.1.1/nightshade-compact-0.1.1.tgz
python3 scripts/validate.py
```

`validate.py` recomputes every count on this page from the recorded outputs and checks their hashes. To rescan the synthetic corpus, follow `REPORT.md` in the pack: install the release into the pack, then run `reproduce.py --stage synthetic` with `--toolchains` pointing at Compact 0.31.1 and 0.34.0 compilers and `--oz-root` at the OpenZeppelin checkout above. The real-contract inputs cannot be redistributed, so only the synthetic results are publicly reproducible.

Pack SHA-256: `9d95e631dc318006610882911c0d58b7c25fa7754c39b49205c309d2e2e57b3b` (1.7 MB).