fips205-slhdsa-verified/verification/extract.sh

82 lines
4.1 KiB
Bash
Raw Normal View History

#!/usr/bin/env bash
# Regenerate the Lean model in gen/ from the pinned fips205 snapshot.
#
# SCOPE: the SLH-DSA verify path, parameter set SLH-DSA-SHA2-128s
# roots: slh_verify(_internal), fors_pk_from_sig, ht_verify,
# xmss_pk_from_sig, wots_pk_from_sig, chain
# (sign/keygen are out of scope; the five verify-path hash oracles
# (h_msg, f, h, t_l, t_len) are opaque — see TRUSTED-BASE.md.)
#
# Rust --charon--> SlhVerify.llbc --aeneas--> gen/SlhVerify/*.lean
#
review round 2: in-Lean exact-cone audit + reproducibility + doc honesty Addresses the round-2 reviewer punch-list. No theorem statement, proof term, or fold definition changed; the eleven cones are unchanged (independent collectAxioms dump in verification/RECORDED-RUN.md). AUDIT GATE (both reviewers, the critical one) - Retire the bash #print-axioms text parser (fail-open on empty/truncated reports, and only a SUBSET check). Replace with verification/Proofs/Audit.lean: reads each certificate's cone from the kernel via collectAxioms and asserts EXACT set equality against its expected boundary. Extra axiom, dropped oracle, renamed/deleted cert, or an axiom/opaque sham each throw -> non-zero Lean exit. No text to misparse; nothing fails open. check.sh Phase 3 now just compiles it (and still requires the explicit PASSED line). - check-selftest.sh rewritten to attack the new gate: dead-file, smuggled extra axiom (named), dropped-oracle (subset would pass, exact must not), and a vanished certificate (the collectAxioms-returns-[] trap). All four rejected. REPRODUCIBILITY (GPT B1.4 / B1.5) - extract.sh refuses a wrong-commit or dirty source tree (fail-closed), takes an optional source-path arg, and pins the source commit. - verification/PROVENANCE.json: single machine-readable pin set (source + charon + aeneas commits/channel + lean + ocaml) with generated-file sha256. - Re-running extract.sh reproduces gen/SlhVerify/{Types,Funs}.lean byte-identically (companion fips205-source commit adds Cargo.lock + rust-toolchain.toml; verified not to perturb the model). DOC HONESTY (both reviewers) - README: fix the self-contradiction (apex "not yet proven" trailer vs the proven apex), the false "oracles kept OUTSIDE every cone" (they are INSIDE, by design), "deployed monomorphic path" and "semantics-identical for every parameter set" overclaims, "only two lines changed", stale snapshot head; retitle the stale future-tense "what will be claimed" section. - TRUSTED-BASE: drop "nothing proven yet"; add base_2b-inner and deployment- bridge non-claims explicitly; current pin. - ChainSpec header: "deployed monomorphic path" -> private verify_mono facade (comment only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:13:55 +00:00
# REPRODUCIBILITY (external review round 2, 2026-07-24): this script now
# REFUSES to extract from a source tree that is not at the pinned commit or
# is dirty (a wrong/uncommitted source would silently produce a different
# model). The full pin set (source + charon + aeneas + lean + ocaml) is in
# verification/PROVENANCE.json; re-running this against the pinned tree
# reproduces gen/SlhVerify/{Types,Funs}.lean byte-identically.
#
# HISTORY (gate-0 finding, resolved 2026-07-22): upstream models the hash
# family as `crate::hashers::Hashers`, a struct of plain function pointers,
review round 2: in-Lean exact-cone audit + reproducibility + doc honesty Addresses the round-2 reviewer punch-list. No theorem statement, proof term, or fold definition changed; the eleven cones are unchanged (independent collectAxioms dump in verification/RECORDED-RUN.md). AUDIT GATE (both reviewers, the critical one) - Retire the bash #print-axioms text parser (fail-open on empty/truncated reports, and only a SUBSET check). Replace with verification/Proofs/Audit.lean: reads each certificate's cone from the kernel via collectAxioms and asserts EXACT set equality against its expected boundary. Extra axiom, dropped oracle, renamed/deleted cert, or an axiom/opaque sham each throw -> non-zero Lean exit. No text to misparse; nothing fails open. check.sh Phase 3 now just compiles it (and still requires the explicit PASSED line). - check-selftest.sh rewritten to attack the new gate: dead-file, smuggled extra axiom (named), dropped-oracle (subset would pass, exact must not), and a vanished certificate (the collectAxioms-returns-[] trap). All four rejected. REPRODUCIBILITY (GPT B1.4 / B1.5) - extract.sh refuses a wrong-commit or dirty source tree (fail-closed), takes an optional source-path arg, and pins the source commit. - verification/PROVENANCE.json: single machine-readable pin set (source + charon + aeneas commits/channel + lean + ocaml) with generated-file sha256. - Re-running extract.sh reproduces gen/SlhVerify/{Types,Funs}.lean byte-identically (companion fips205-source commit adds Cargo.lock + rust-toolchain.toml; verified not to perturb the model). DOC HONESTY (both reviewers) - README: fix the self-contradiction (apex "not yet proven" trailer vs the proven apex), the false "oracles kept OUTSIDE every cone" (they are INSIDE, by design), "deployed monomorphic path" and "semantics-identical for every parameter set" overclaims, "only two lines changed", stale snapshot head; retitle the stale future-tense "what will be claimed" section. - TRUSTED-BASE: drop "nothing proven yet"; add base_2b-inner and deployment- bridge non-claims explicitly; current pin. - ChainSpec header: "deployed monomorphic path" -> private verify_mono facade (comment only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:13:55 +00:00
# which Aeneas cannot translate. The compat patch in fips205-source provides
# the additive monomorphic verify_mono module whose hash suite is reached
# through named free functions — this script roots there.
#
review round 2: in-Lean exact-cone audit + reproducibility + doc honesty Addresses the round-2 reviewer punch-list. No theorem statement, proof term, or fold definition changed; the eleven cones are unchanged (independent collectAxioms dump in verification/RECORDED-RUN.md). AUDIT GATE (both reviewers, the critical one) - Retire the bash #print-axioms text parser (fail-open on empty/truncated reports, and only a SUBSET check). Replace with verification/Proofs/Audit.lean: reads each certificate's cone from the kernel via collectAxioms and asserts EXACT set equality against its expected boundary. Extra axiom, dropped oracle, renamed/deleted cert, or an axiom/opaque sham each throw -> non-zero Lean exit. No text to misparse; nothing fails open. check.sh Phase 3 now just compiles it (and still requires the explicit PASSED line). - check-selftest.sh rewritten to attack the new gate: dead-file, smuggled extra axiom (named), dropped-oracle (subset would pass, exact must not), and a vanished certificate (the collectAxioms-returns-[] trap). All four rejected. REPRODUCIBILITY (GPT B1.4 / B1.5) - extract.sh refuses a wrong-commit or dirty source tree (fail-closed), takes an optional source-path arg, and pins the source commit. - verification/PROVENANCE.json: single machine-readable pin set (source + charon + aeneas commits/channel + lean + ocaml) with generated-file sha256. - Re-running extract.sh reproduces gen/SlhVerify/{Types,Funs}.lean byte-identically (companion fips205-source commit adds Cargo.lock + rust-toolchain.toml; verified not to perturb the model). DOC HONESTY (both reviewers) - README: fix the self-contradiction (apex "not yet proven" trailer vs the proven apex), the false "oracles kept OUTSIDE every cone" (they are INSIDE, by design), "deployed monomorphic path" and "semantics-identical for every parameter set" overclaims, "only two lines changed", stale snapshot head; retitle the stale future-tense "what will be claimed" section. - TRUSTED-BASE: drop "nothing proven yet"; add base_2b-inner and deployment- bridge non-claims explicitly; current pin. - ChainSpec header: "deployed monomorphic path" -> private verify_mono facade (comment only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:13:55 +00:00
# Usage: ./extract.sh [PATH_TO_fips205-source]
# (default: ~/GitClone/FormalVerification/sources/fips205-source)
# Override the required commit only for a deliberate re-pin:
# EXPECTED_SRC_COMMIT=<full-sha> ./extract.sh [PATH]
set -euo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
review round 2: in-Lean exact-cone audit + reproducibility + doc honesty Addresses the round-2 reviewer punch-list. No theorem statement, proof term, or fold definition changed; the eleven cones are unchanged (independent collectAxioms dump in verification/RECORDED-RUN.md). AUDIT GATE (both reviewers, the critical one) - Retire the bash #print-axioms text parser (fail-open on empty/truncated reports, and only a SUBSET check). Replace with verification/Proofs/Audit.lean: reads each certificate's cone from the kernel via collectAxioms and asserts EXACT set equality against its expected boundary. Extra axiom, dropped oracle, renamed/deleted cert, or an axiom/opaque sham each throw -> non-zero Lean exit. No text to misparse; nothing fails open. check.sh Phase 3 now just compiles it (and still requires the explicit PASSED line). - check-selftest.sh rewritten to attack the new gate: dead-file, smuggled extra axiom (named), dropped-oracle (subset would pass, exact must not), and a vanished certificate (the collectAxioms-returns-[] trap). All four rejected. REPRODUCIBILITY (GPT B1.4 / B1.5) - extract.sh refuses a wrong-commit or dirty source tree (fail-closed), takes an optional source-path arg, and pins the source commit. - verification/PROVENANCE.json: single machine-readable pin set (source + charon + aeneas commits/channel + lean + ocaml) with generated-file sha256. - Re-running extract.sh reproduces gen/SlhVerify/{Types,Funs}.lean byte-identically (companion fips205-source commit adds Cargo.lock + rust-toolchain.toml; verified not to perturb the model). DOC HONESTY (both reviewers) - README: fix the self-contradiction (apex "not yet proven" trailer vs the proven apex), the false "oracles kept OUTSIDE every cone" (they are INSIDE, by design), "deployed monomorphic path" and "semantics-identical for every parameter set" overclaims, "only two lines changed", stale snapshot head; retitle the stale future-tense "what will be claimed" section. - TRUSTED-BASE: drop "nothing proven yet"; add base_2b-inner and deployment- bridge non-claims explicitly; current pin. - ChainSpec header: "deployed monomorphic path" -> private verify_mono facade (comment only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:13:55 +00:00
CRATE="${1:-$HOME/GitClone/FormalVerification/sources/fips205-source}"
# The pinned source commit this repo's model + proofs were verified against.
# Keep in lockstep with verification/PROVENANCE.json and the README snapshot.
round 7: assert pin-map completeness (NEW-13), correct the count to 137, fix the regeneration-scope contradiction The third reviewer demonstrated NEW-13: PROVENANCE.json is a tracked file that nothing pins, and Phase 0's only completeness test was 'is the map non-empty'. Deleting the harness_integrity_sha256 key therefore silently un-pinned BOTH lean-guard and Proofs/Audit.lean with no diagnostic, after which the round-6 logic mutation ran to ALL GREEN over a repository proving False with the digest byte-identical. Reproduced here before fixing. The required pin NAMES now live hardcoded in check.sh — policy in the root of trust, values in the map — so a shortened map is a build failure naming the missing entries. Self-test attack 18 performs the deletion. GPT reviewer, independently: the documented '131 assertion points' was wrong. Recounted from the code, the defensible figure is 137 mono-path evaluated cases (9 retained original + 108 randomized + 10 NIST internal + 10 NIST external-pure); 131 had folded in 3 deployed-only prehash cases while omitting the retained test, and TRUSTED-BASE then decomposed it as 20 + 108 = 128, contradicting itself. Item 9 now carries the full table, states that 127 of the 137 compare mono against deployed, keeps the 3 prehash cases explicitly outside the total, and records that only two SHA2-512 and one SHAKE-256 vector are executable there — so this is not NIST coverage of all four supported prehash variants. Also from GPT: PROVENANCE.json contradicted itself, saying extraction 'reproduces all four model files byte-identically' while its own _comment correctly said the two *External files are hand-maintained. Extraction regenerates two files; the other two are byte-pinned. Corrected. TRUSTED-BASE item 11 now discloses that PROVENANCE.json is itself load-bearing and unpinned, and item 7's stale snapshot head is fixed. README states the lean-guard graceful fallback and that the empirical bridge runs on stable Rust without any Lean toolchain (round-7 NEW-16), which is the first load-bearing part of this work a third party can reproduce with cargo alone. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-28 11:03:35 +00:00
EXPECTED_SRC_COMMIT="${EXPECTED_SRC_COMMIT:-c945821bf964a78bed3ba920ad5aa021083e6237}"
review round 2: in-Lean exact-cone audit + reproducibility + doc honesty Addresses the round-2 reviewer punch-list. No theorem statement, proof term, or fold definition changed; the eleven cones are unchanged (independent collectAxioms dump in verification/RECORDED-RUN.md). AUDIT GATE (both reviewers, the critical one) - Retire the bash #print-axioms text parser (fail-open on empty/truncated reports, and only a SUBSET check). Replace with verification/Proofs/Audit.lean: reads each certificate's cone from the kernel via collectAxioms and asserts EXACT set equality against its expected boundary. Extra axiom, dropped oracle, renamed/deleted cert, or an axiom/opaque sham each throw -> non-zero Lean exit. No text to misparse; nothing fails open. check.sh Phase 3 now just compiles it (and still requires the explicit PASSED line). - check-selftest.sh rewritten to attack the new gate: dead-file, smuggled extra axiom (named), dropped-oracle (subset would pass, exact must not), and a vanished certificate (the collectAxioms-returns-[] trap). All four rejected. REPRODUCIBILITY (GPT B1.4 / B1.5) - extract.sh refuses a wrong-commit or dirty source tree (fail-closed), takes an optional source-path arg, and pins the source commit. - verification/PROVENANCE.json: single machine-readable pin set (source + charon + aeneas commits/channel + lean + ocaml) with generated-file sha256. - Re-running extract.sh reproduces gen/SlhVerify/{Types,Funs}.lean byte-identically (companion fips205-source commit adds Cargo.lock + rust-toolchain.toml; verified not to perturb the model). DOC HONESTY (both reviewers) - README: fix the self-contradiction (apex "not yet proven" trailer vs the proven apex), the false "oracles kept OUTSIDE every cone" (they are INSIDE, by design), "deployed monomorphic path" and "semantics-identical for every parameter set" overclaims, "only two lines changed", stale snapshot head; retitle the stale future-tense "what will be claimed" section. - TRUSTED-BASE: drop "nothing proven yet"; add base_2b-inner and deployment- bridge non-claims explicitly; current pin. - ChainSpec header: "deployed monomorphic path" -> private verify_mono facade (comment only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-24 17:13:55 +00:00
# ── Provenance guard: refuse a wrong or dirty source tree (fail-closed) ──────
[ -d "$CRATE/.git" ] || { echo "ERROR: '$CRATE' is not a git checkout of fips205-source." >&2; exit 2; }
GOT_COMMIT="$(git -C "$CRATE" rev-parse HEAD)"
if [ "$GOT_COMMIT" != "$EXPECTED_SRC_COMMIT" ]; then
echo "ERROR: source is at ${GOT_COMMIT:0:12}, but this repo is pinned to" >&2
echo " ${EXPECTED_SRC_COMMIT:0:12}. Check out the pin, or set" >&2
echo " EXPECTED_SRC_COMMIT=<sha> for a deliberate re-pin." >&2
exit 3
fi
if [ -n "$(git -C "$CRATE" status --porcelain)" ]; then
echo "ERROR: source tree at '$CRATE' is dirty. Extraction must run against" >&2
echo " a clean, committed tree so the model is reproducible." >&2
git -C "$CRATE" status --porcelain | sed 's/^/ /' >&2
exit 4
fi
echo "[0/2] provenance OK: fips205-source @ ${EXPECTED_SRC_COMMIT:0:12} (clean)"
# Toolchain env is sourced only AFTER the provenance guard, so a party without
# the aeneas toolchain still gets the clean exit-2/3/4 diagnostics above
# (round-3 reviewer nit, 2026-07-24).
source ~/aeneas-toolchain/env.sh
echo "[1/2] charon: Rust -> LLBC (monomorphic SHA2-128s verify cone;"
echo " crate::verify_mono::oracle is the opaque SHA-2 boundary)"
cd "$CRATE"
# Single extraction root: the monomorphic entry. The five hash primitives
# are reached through crate::verify_mono::oracle, marked opaque here — this
# is the deliberate SHA-2 trust boundary (documented in TRUSTED-BASE.md).
# The generic Hashers fn-pointer path is NOT in this cone by construction.
charon cargo --preset=aeneas \
--start-from 'crate::verify_mono::slh_verify_128s' \
--opaque 'crate::verify_mono::oracle' \
--opaque 'sha2' --opaque 'sha3' --opaque 'zeroize' --opaque 'rand_core' \
--hide-marker-traits \
--dest-file "$HERE/SlhVerify.llbc" \
-- --no-default-features --features slh_dsa_sha2_128s
echo "[2/2] aeneas: LLBC -> Lean (split files, SlhVerify.* modules;"
echo " hand-maintained TypesExternal.lean / FunsExternal.lean are"
echo " NOT overwritten once they exist)"
cd "$HERE"
aeneas -backend lean -split-files -subdir SlhVerify -dest gen SlhVerify.llbc
echo "Done. Now run ./check.sh (Phase 1: the regenerated model must type-check)."