mirror of
https://github.com/saymrwulf/ltl-accumulator-verified.git
synced 2026-09-06 20:20:48 +00:00
Round-2 external reviews (GPT-5.6 + second Claude) converged on the coverage gate being evadable (H1/NEW-1); GPT additionally proved the kit's fidelity target could not run (H2) and the namespace-collision attack that defeats any source-regex fix. This round adopts GPT's required correction in full: - Proofs/Inventory.lean: declaration inventory read from the compiled Lean environment — every constant of every corpus module, fully qualified, unfiltered (compiler auxiliaries and _private mangles pinned too), with kind and axiom cone; own cone walker cross-checked in-process against core collectAxioms (hard error on divergence). - verification/inventory-allowlist.txt: all 218 constants pinned. - inventory_gate.sh: fail-closed diff both directions (UNCLASSIFIED / STALE), INV-COUNT truncation guard, exactly-one-axiom invariant. - check.sh Phase 3b rewritten around the gate + manifest⇔inventory drift checks + CONES⇔inventory cone cross-check (two independent computations must agree). EXCLUDE table gone (sha256/Bytes are ordinary audited entries now). - selftest_audit.sh: 9 adversarial cases against the production gate (attributed/indented/private/instance, namespace collision, smuggled axiom, deleted decl, unmanifested Proofs/ and gen/ modules) + positive control — all defeated (GPT release condition 2). - M1: recursive orphan-olean guard (caught a stray dev artifact on its first run), gen/ dead-file check, corpus-wide single-axiom pin. - L1/NEW-2: acceptIncl_sound drops the redundant hm (derived from hacc.1); cone unchanged. - M2/M3: STATEMENT-MAP counts 230,271/230,016; non-vacuity guard wording narrowed to what the guards actually certify. - README layer table: stale L4/pin-store rows fixed (missed by both round-2 reviewers AND the round-2 revision — found in self-review). - KNOWN-GAPS 12 (audit-gate lineage + residual limits), 13 (round-2 kit target not self-contained); gap 2 count fixed. - RESPONSE-TO-REVIEWERS.md: round-3 disposition of every finding. Kit round 3 additionally ships the complete stdlib-only import closure of pacta.transparency (content-addressed vs pacta 3d81d53), the clean-extraction fidelity transcript (exit 0, 230,271+230,016, zero mismatches), the ATTESTATION GREEN check.sh transcript, and the self-test transcript. The live LTL remains untouched (12 leaves, root bcd15f9d…); attestation stays blocked pending ePrint decision + author review + explicit operator order. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
136 lines
7.1 KiB
Markdown
136 lines
7.1 KiB
Markdown
# Response to reviewers — round 3
|
|
|
|
Corpus: `ltl-accumulator-verified`. Round-2 reviews received against the
|
|
round-2 freeze `260ad64` (GPT-5.6 second adversarial review; second
|
|
Claude round-2 findings). Every finding was independently re-verified
|
|
against the corpus before any change; all confirmed findings are fixed
|
|
in this freeze. No theorem statement changed except the one interface
|
|
tightening both reviewers requested (L1/NEW-2).
|
|
|
|
## Disposition of round-2 findings
|
|
|
|
### GPT H1 / Claude NEW-1 — coverage gate evadable (CONFIRMED, fixed structurally)
|
|
|
|
Both reviewers were right, and GPT's namespace-collision attack
|
|
(`LTLAcc.Hidden.MTH` classified against `CONES[LTLAcc.MTH]`) showed that
|
|
regex hardening (Claude's proposed fix) would not have closed the class.
|
|
We adopted GPT's required correction in full — the inventory is now
|
|
derived from the Lean environment, not from source:
|
|
|
|
- **`Proofs/Inventory.lean`** imports every corpus module and emits
|
|
EVERY constant whose originating module is a corpus module: fully
|
|
qualified name, declaration kind, and axiom cone. There is **no
|
|
filtering** — compiler-generated auxiliaries and `_private.*` mangles
|
|
are emitted and pinned too, so there is no name shape that can hide.
|
|
A corpus module missing from the import list is an elaboration error.
|
|
- The axiom cone is computed by our own walker AND cross-checked
|
|
in-process against core `collectAxioms` (the machinery `#print axioms`
|
|
uses) for every constant — divergence is a hard error. (This caught a
|
|
real toolchain subtlety during development: `ConstantInfo.value?`
|
|
returns `none` for theorems on 4.30.0-rc2, which would have silently
|
|
truncated cones; the direct constructor match avoids it, and the
|
|
cross-check would have refused to ship it.)
|
|
- **`verification/inventory-allowlist.txt`** pins all 218 constants;
|
|
**`inventory_gate.sh`** diffs environment vs allowlist fail-closed in
|
|
BOTH directions (UNCLASSIFIED / STALE), requires the INV-COUNT
|
|
trailer (a truncated Lean run cannot pass as an empty diff), and
|
|
asserts the whole corpus contains exactly one axiom-kind constant:
|
|
`LTLAcc.sha256`.
|
|
- check.sh Phase 3b additionally verifies: the inventory's module list
|
|
== the compile manifest (both directions), every CONES entry appears
|
|
in the allowlist **with an identical cone** (two independent cone
|
|
computations must agree), and every CONES entry is queried by
|
|
AxiomCheck.
|
|
- **GPT release condition 2 (adversarial tests) is met by
|
|
`verification/selftest_audit.sh`**, which attacks the exact production
|
|
gate: attributed, indented, private, and instance declarations, the
|
|
nested-namespace basename collision, a smuggled axiom, a deleted
|
|
declaration (STALE direction), and unmanifested `Proofs/` and `gen/`
|
|
modules through the full check.sh — plus a positive control so the
|
|
self-test cannot pass vacuously. Transcript in the kit
|
|
(`selftest-transcript.txt`).
|
|
|
|
### GPT H2 — fidelity target could not run (CONFIRMED, fixed)
|
|
|
|
Reproduced exactly (`ModuleNotFoundError: pacta.postquantum`). The
|
|
round-3 `pacta-fidelity-target` ships the complete **load-time import
|
|
closure** of `pacta.transparency` (`__init__`, `transparency`,
|
|
`postquantum`, `signing`, `yamlio`) — all stdlib-only, so a bare
|
|
Python 3 runs it with no pip installs — content-addressed in
|
|
`MANIFEST.sha256` against pacta commit `3d81d538…` with verification
|
|
instructions (`TARGET-PROVENANCE.md`). The kit includes the complete
|
|
terminal transcript AND exit code of `run_fidelity.py` executed from a
|
|
clean extraction: exit 0, `230,271 + 230,016`, zero mismatches
|
|
(`fidelity-clean-run-transcript.txt`), plus the full green `check.sh`
|
|
transcript ending in `ATTESTATION GREEN` with fidelity not skipped
|
|
(`check-transcript.txt`) — GPT release conditions 3 and 5.
|
|
|
|
### GPT M1 — audit narrower outside Proofs/ (CONFIRMED, fixed)
|
|
|
|
- Orphan-olean guard is now recursive over the whole tree (it caught a
|
|
stray development artifact on its first run).
|
|
- gen/ has the same unmanifested-source ("dead file") check as Proofs/.
|
|
- The axiom surface is pinned corpus-wide twice: textually (exactly one
|
|
`axiom` line under gen/, none under Proofs/) and semantically (the
|
|
inventory admits exactly one axiom-kind constant anywhere).
|
|
- Declaration discovery under gen/ now goes through the environment
|
|
inventory like everything else.
|
|
|
|
### GPT M2 — stale fidelity counts in STATEMENT-MAP (CONFIRMED, fixed)
|
|
|
|
The row now reads 230,271 + 230,016 with the expanded families named.
|
|
Process note: the round-2 Claude review certified this row as already
|
|
fixed; it was not. Consistent with this project's experience, "verified"
|
|
claims by reviewers are themselves re-verified now.
|
|
|
|
### GPT M3 — "not choice-dischargeable" overclaims (CONFIRMED, fixed)
|
|
|
|
STATEMENT-MAP now uses (essentially) GPT's safer wording: the guards
|
|
show each named extractor returns a non-collision on at least one
|
|
canonical honest input, ruling out the globally-inhabited-existential
|
|
degeneration; they do NOT establish logical dependence on every
|
|
hypothesis, nor exclude other classical arguments on restricted domains.
|
|
|
|
### GPT L1 / Claude NEW-2 — redundant `hm` in `acceptIncl_sound` (CONFIRMED, fixed)
|
|
|
|
The hypothesis is gone; the range fact is derived from `hacc.1`, so the
|
|
theorem is stated purely in terms of acceptance + wrong-leaf premise.
|
|
Cone unchanged (`propext, Classical.choice, LTLAcc.sha256, Quot.sound`),
|
|
re-verified by `#print axioms` and the inventory.
|
|
|
|
### GPT ledger additions (adopted, as history)
|
|
|
|
KNOWN-GAPS gains gap 12 (audit-gate lineage: the round-2 gate was
|
|
evadable, what replaced it, and the residual limits of an
|
|
environment-derived inventory — it cannot see never-compiled source,
|
|
which the dead-file checks cover, nor defeat a hostile toolchain) and
|
|
gap 13 (the round-2 kit's fidelity target was not self-contained).
|
|
|
|
## Found in round-3 self-review (neither reviewer caught)
|
|
|
|
- **README layer table was still stale at `260ad64`**: L4 carried
|
|
"queued for S4 restoration" and the pin-store row said "pending",
|
|
despite round-1 F2 being certified as fixed by the round-2 Claude
|
|
review. The table now matches the frozen state (all layers done,
|
|
L4 explicitly "done as specializations" per gap 3).
|
|
- KNOWN-GAPS gap 2 still cited the old 164,224 count; updated.
|
|
|
|
## Round-2 Claude review, remaining notes
|
|
|
|
NEW-1 was correct in direction; we implemented the stronger
|
|
environment-based fix rather than the proposed regex broadening, since
|
|
the namespace collision defeats any basename-keyed source scan. The
|
|
round-2 Claude claim that the fidelity harness was "confirmed green
|
|
against the real pacta code" was obtained by hand-stubbing the missing
|
|
modules — with the round-3 self-contained target, that result is now
|
|
reproducible by anyone from the kit alone.
|
|
|
|
## What did NOT change
|
|
|
|
All theorem statements and proofs except the `acceptIncl_sound`
|
|
signature tightening; the axiom boundary (single opaque `sha256`); the
|
|
pinned CONES table (59 entries, all cones byte-identical to round 2);
|
|
the fidelity counts. The live transparency log remains frozen at
|
|
12 leaves (root `bcd15f9d…`) and is untouched by this round;
|
|
attestation remains blocked pending the ePrint decision, author
|
|
review, and an explicit operator order.
|