mirror of
https://github.com/saymrwulf/ltl-accumulator-verified.git
synced 2026-09-04 20:03:44 +00:00
verification: pin the harness, audit drivers and policy files (P1-c)
This repository has the estate's strongest gates, which makes them the most valuable to switch off. Until now every one of them was executed by scripts that nothing pinned. Phase 0c requires every harness file to match HARNESS.sha256 — 15 files: check.sh, lean-guard, inventory_gate.sh, run_bare.sh, all three self-tests, both audit drivers (Proofs/Inventory.lean, Proofs/AxiomCheck.lean), the policy tables (inventory-allowlist.txt, AUDIT-MANIFEST.txt), the toolchain pin, the fidelity harness and its Python transcription, and the extracted model. WHICH files must be pinned is policy and lives in check.sh, never in the map being consulted: the required set is derived from the filesystem (the executable bit, plus gen/**.lean, plus an explicit list for the rest), so deleting a pin entry is a set mismatch rather than a silent un-pinning. gen/LTLAcc/HashExternal.lean was previously bound by nothing at all — it was compiled and trusted. It is now pinned, and the derivation is by set, so a new model file fails closed. selftest_audit.sh case 9 is split rather than relabelled. Phase 0c now catches an unpinned rogue gen module BEFORE the dead-file gate runs, so asserting only the new diagnostic would have quietly retired the dead-file gate from the test suite. 9a asserts the harness-set mismatch on the unpinned file; 9b pins it — an author who added it deliberately — and asserts Phase 2 still dies with DEAD FILE (gen). Ten cases now, all defeated. KNOWN-GAPS and the trusted base record the circularity plainly: an author who edits a script and refreshes its pin in one commit passes every phase. The pin removes the silent path, not the possibility. Review at the pinned commit remains the consumer's protection. Verified green after the fix: button (75s), harness self-test, binding self-test, and the ten-case audit self-test. ATTESTATION GREEN (Lean + fidelity), all fidelity case counts identical to the pre-change run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
b575e857e3
commit
8530cc73f4
4 changed files with 199 additions and 7 deletions
15
verification/HARNESS.sha256
Normal file
15
verification/HARNESS.sha256
Normal file
|
|
@ -0,0 +1,15 @@
|
|||
e7d422f0be9a9e6f5465058292e30c711d52856428da2b01c470a57ec181540c AUDIT-MANIFEST.txt
|
||||
7ab1bfbafc8ac9ab34d1e0eaa71ba2f1f9912121345cc1af281bcb1d2ec30ad2 check.sh
|
||||
070147e2667053bd5d5e1174b969fc6c91bfcf15ded1a5bff57754e15f416885 fidelity/lean_defs.py
|
||||
5d82462a002ac9fc782e95afe78b7719ba64b6410b5d2bfa620fe5317367dbf2 fidelity/run_fidelity.py
|
||||
503babb3f4e6aff82ebd59e8752469ecd60fba440ed3b11b3f97c2b655fbd9bf gen/LTLAcc/HashExternal.lean
|
||||
f1eb5cdd158e30df14c59065fe2050448c77b5262b282208fa831d050f6b6a71 inventory-allowlist.txt
|
||||
f66fb98d2a09503d9bd0d60dc964545eea6dc94b9bbb9246d1021195b79f2601 inventory_gate.sh
|
||||
736ea4be712e1b5bcda10ecb466f0dec7008a2a36eabdfd77563976299c43cce lean-guard
|
||||
ce4c4e3d87434b9663f46de25ce34b48a0cf0d392e0a320a0787b4674a2d7b61 lean-toolchain
|
||||
eda93f520546a692926b2a46bcb79332e1795879e5083327a8bf2404aca5cf87 Proofs/AxiomCheck.lean
|
||||
25344c8441077277addd6b4eb769b2c0e3b173afc28d3a6e8aeba0580c22a87f Proofs/Inventory.lean
|
||||
0b8a0fc6947af1d1e600a756eb2b07dc88d189b21df3220c4501a41be8b33f1e run_bare.sh
|
||||
67a44e0db818efc48dede26f73f76f1221424bb23e40e29d859b12d0d300403b selftest_audit.sh
|
||||
3d5898161d663eccad162269a5a6c102319077e22e1f2d89a8bfcab6926d29f6 selftest-harness.sh
|
||||
cf6d4d8210e224a054d4ab693c28c83e7a9ddebda05da47d6ec311d825a606c0 selftest_statements.sh
|
||||
|
|
@ -116,6 +116,59 @@ while IFS= read -r -d '' o; do
|
|||
[ -f "${o%.olean}.lean" ] || { echo "ORPHAN OLEAN: $o has no sibling .lean (stale artifact)"; exit 1; }
|
||||
done < <(find "$HERE" -name '*.olean' -print0)
|
||||
|
||||
# ── Phase 0c: harness integrity ─────────────────────────────────────────────
|
||||
# WHY. Every gate below is executed by a script that, until now, nothing
|
||||
# pinned. Round-5 review of the companion SLH-DSA repository stubbed the
|
||||
# compiler wrapper alone and its button printed ALL GREEN in 3.6 seconds over
|
||||
# deliberately destroyed proofs. Depth of checking is worth nothing if the
|
||||
# thing doing the checking is unbound — and this repo's gates are the estate's
|
||||
# strongest, which makes them the most valuable to switch off.
|
||||
#
|
||||
# WHICH files must be pinned is POLICY, and policy lives here — in the root of
|
||||
# trust — never inside the map being consulted. If the required set were read
|
||||
# from HARNESS.sha256, deleting an entry would silently un-pin that file
|
||||
# instead of failing the build.
|
||||
#
|
||||
# Membership is SELF-DERIVING from two sources the filesystem can answer: the
|
||||
# executable bit (anything this script can shell out to) and gen/**.lean (the
|
||||
# extracted model, which nothing else byte-pins in this repo). Load-bearing
|
||||
# files that are neither — the audit drivers, the policy tables, the toolchain
|
||||
# pin, the fidelity harness — cannot be discovered and are listed explicitly.
|
||||
HARNESS_EXTRA=(
|
||||
AUDIT-MANIFEST.txt # the statement block Phase 3d's digest is taken over
|
||||
inventory-allowlist.txt # the pinned audit surface Phase 3b diffs against
|
||||
lean-toolchain # which Lean the corpus claims to have been checked by
|
||||
fidelity/lean_defs.py # the Python transcription the differential compares
|
||||
fidelity/run_fidelity.py # the differential itself
|
||||
Proofs/Inventory.lean # audit driver: emits the inventory AND the statements
|
||||
Proofs/AxiomCheck.lean # audit driver: the #print axioms queries of Phase 3
|
||||
)
|
||||
echo "=== Phase 0c: harness integrity ==="
|
||||
if [ ! -s "$HERE/HARNESS.sha256" ]; then
|
||||
echo "FATAL: HARNESS.sha256 is missing or empty — the harness is unpinned."
|
||||
exit 1
|
||||
fi
|
||||
# check.sh is pinned like everything else: that catches drift and accident. It
|
||||
# does NOT stop an author who edits this script and refreshes its pin in one
|
||||
# commit — nothing executed by the harness can. The defence there is that both
|
||||
# changes appear in the diff at the pinned commit.
|
||||
HARNESS_REQUIRED=$( { find "$HERE" -type f -executable -not -path '*/.git/*' -printf '%P\n'
|
||||
find "$HERE/gen" -type f -name '*.lean' -printf 'gen/%P\n'
|
||||
printf '%s\n' "${HARNESS_EXTRA[@]}"; } | sort -u )
|
||||
HARNESS_PINNED=$(awk '{print $2}' "$HERE/HARNESS.sha256" | sort -u)
|
||||
if [ "$HARNESS_REQUIRED" != "$HARNESS_PINNED" ]; then
|
||||
echo "FATAL: the set of harness files does not match HARNESS.sha256."
|
||||
echo " (< pinned, > present and requiring a pin)"
|
||||
diff <(echo "$HARNESS_PINNED") <(echo "$HARNESS_REQUIRED") | sed 's/^/ /'
|
||||
exit 1
|
||||
fi
|
||||
if ! ( cd "$HERE" && sha256sum -c --quiet HARNESS.sha256 ) ; then
|
||||
echo "FATAL: a harness file does not match its pin. The button you are"
|
||||
echo "running is not the button that was reviewed."
|
||||
exit 1
|
||||
fi
|
||||
echo " $(wc -l < "$HERE/HARNESS.sha256") harness files match their pins"
|
||||
|
||||
echo "=== Phase 1: stub + axiom-smuggling audit ==="
|
||||
if grep -rn 'by trivial' "$HERE"/Proofs/*.lean 2>/dev/null; then
|
||||
echo "STUB DETECTED"; exit 1; fi
|
||||
|
|
|
|||
108
verification/selftest-harness.sh
Executable file
108
verification/selftest-harness.sh
Executable file
|
|
@ -0,0 +1,108 @@
|
|||
#!/usr/bin/env bash
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
# selftest-harness.sh — adversarial self-test for check.sh Phase 0c.
|
||||
#
|
||||
# Phase 0c pins the scripts and policy files the button itself runs on. The
|
||||
# attack it exists to stop is the cheapest one in the estate: don't touch the
|
||||
# proofs at all, edit the checker. Round-5 review of the companion SLH-DSA
|
||||
# repository stubbed the compiler wrapper alone and got ALL GREEN in 3.6
|
||||
# seconds over deliberately destroyed proofs.
|
||||
#
|
||||
# Cases, each asserting a SPECIFIC diagnostic:
|
||||
# 0 positive control: untouched tree passes
|
||||
# 1 a pinned harness file edited by one byte → does not match its pin
|
||||
# 2 a NEW executable appears, unpinned → set mismatch
|
||||
# 3 an entry DELETED from HARNESS.sha256 → set mismatch, NOT a
|
||||
# silent un-pin (this is the shape of the defect SLH-DSA round-6 found:
|
||||
# dropping a key un-pinned two files with no diagnostic at all)
|
||||
# 4 HARNESS.sha256 itself removed → fail-closed
|
||||
#
|
||||
# Phase 0c is lifted out of check.sh at run time, so the tested logic IS the
|
||||
# shipping logic. Cheap: no Lean, runs in about a second.
|
||||
# ─────────────────────────────────────────────────────────────────────────────
|
||||
set -uo pipefail
|
||||
HERE="$(cd "$(dirname "$0")" && pwd)"
|
||||
FAILURES=0
|
||||
STASH="$(mktemp -d)"
|
||||
NEWEXE="$HERE/zz-selftest-helper.sh"
|
||||
|
||||
cleanup() {
|
||||
[ -f "$STASH/HARNESS.sha256" ] && cp "$STASH/HARNESS.sha256" "$HERE/HARNESS.sha256"
|
||||
[ -f "$STASH/victim" ] && cp "$STASH/victim" "$HERE/$VICTIM"
|
||||
rm -f "$NEWEXE"
|
||||
rm -rf "$STASH"
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
cp "$HERE/HARNESS.sha256" "$STASH/HARNESS.sha256"
|
||||
|
||||
# Lift Phase 0c. The two repo families end the phase differently, so accept
|
||||
# either terminator rather than hardcoding one and silently lifting nothing.
|
||||
DRIVER="$STASH/phase0c.sh"
|
||||
{
|
||||
echo 'set -uo pipefail'
|
||||
echo "HERE=\"$HERE\""
|
||||
awk '/^# ── Phase 0c/{f=1} f{print} /^# ── Phase 1|^echo "=== Phase 1/{if(f && !/Phase 0c/) exit}' "$HERE/check.sh" \
|
||||
| sed '/^# ── Phase 1/d; /^echo "=== Phase 1/d'
|
||||
} > "$DRIVER"
|
||||
if [ "$(grep -c . "$DRIVER")" -lt 20 ]; then
|
||||
echo "FATAL: could not lift Phase 0c out of check.sh — the phase markers moved."
|
||||
echo "This self-test must attack the shipping gate; refusing to run against nothing."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
expect() { # expect <label> <want-rc> <want-substring>
|
||||
local label="$1" want_rc="$2" want_txt="$3" out rc
|
||||
out=$(bash "$DRIVER" 2>&1); rc=$?
|
||||
if [ "$rc" -ne "$want_rc" ]; then
|
||||
echo " ✗ $label: exit $rc, expected $want_rc"; echo "$out" | sed 's/^/ /'
|
||||
FAILURES=$((FAILURES+1)); return
|
||||
fi
|
||||
if ! grep -qF "$want_txt" <<<"$out"; then
|
||||
echo " ✗ $label: exit code right, diagnostic wrong (rejected for the wrong reason)"
|
||||
echo " wanted: $want_txt"; echo "$out" | sed 's/^/ /'
|
||||
FAILURES=$((FAILURES+1)); return
|
||||
fi
|
||||
echo " ✓ $label"
|
||||
}
|
||||
|
||||
echo "=== selftest-harness: attacking check.sh Phase 0c ==="
|
||||
|
||||
# ── 0. positive control ────────────────────────────────────────────────────
|
||||
expect "case 0 control: untouched harness passes" 0 "match their pins"
|
||||
|
||||
# ── 1. edit a pinned file. lean-guard is the pointed choice: it is the memory
|
||||
# cap protecting this machine, and stubbing it is the demonstrated
|
||||
# 3.6-second path to a false green.
|
||||
VICTIM=lean-guard
|
||||
cp "$HERE/$VICTIM" "$STASH/victim"
|
||||
printf '\n# selftest\n' >> "$HERE/$VICTIM"
|
||||
expect "case 1: edited lean-guard caught" 1 "does not match its pin"
|
||||
cp "$STASH/victim" "$HERE/$VICTIM"
|
||||
|
||||
# ── 2. a new executable the button could shell out to ──────────────────────
|
||||
printf '#!/bin/sh\necho "unpinned"\n' > "$NEWEXE"; chmod +x "$NEWEXE"
|
||||
expect "case 2: new unpinned executable caught" 1 "does not match HARNESS.sha256"
|
||||
rm -f "$NEWEXE"
|
||||
|
||||
# ── 3. delete a pin entry: the set must be derived from the filesystem, not
|
||||
# read out of the map being consulted, or this is a silent un-pin.
|
||||
grep -v " ${VICTIM}\$" "$STASH/HARNESS.sha256" > "$HERE/HARNESS.sha256"
|
||||
expect "case 3: deleted pin entry is a failure, not a silent un-pin" 1 "does not match HARNESS.sha256"
|
||||
cp "$STASH/HARNESS.sha256" "$HERE/HARNESS.sha256"
|
||||
|
||||
# ── 4. absence must not pass for cleanliness ───────────────────────────────
|
||||
rm -f "$HERE/HARNESS.sha256"
|
||||
expect "case 4: missing pin file is fail-closed" 1 "the harness is unpinned"
|
||||
cp "$STASH/HARNESS.sha256" "$HERE/HARNESS.sha256"
|
||||
|
||||
# ── 5. restored ────────────────────────────────────────────────────────────
|
||||
expect "case 5: restored to green" 0 "match their pins"
|
||||
|
||||
echo ""
|
||||
if [ "$FAILURES" -eq 0 ]; then
|
||||
echo "SELFTEST PASSED — Phase 0c rejects harness tampering for the stated reason."
|
||||
exit 0
|
||||
fi
|
||||
echo "SELFTEST FAILED: $FAILURES case(s) did not behave as claimed."
|
||||
exit 1
|
||||
|
|
@ -123,15 +123,31 @@ grep -q "DEAD FILE: Proofs/Rogue.lean" "$T/check8.out" || {
|
|||
echo " ✓ case 8 unmanifested Proofs module: check.sh dies with DEAD FILE"
|
||||
rm -f "$T/Proofs/Rogue.lean"
|
||||
|
||||
# 9 — unmanifested gen/ module (full check.sh; dies in Phase 2)
|
||||
# 9 — unmanifested gen/ module. TWO gates stand here now and the case
|
||||
# exercises BOTH, because asserting only the outer one would quietly
|
||||
# retire the inner one from the test suite.
|
||||
#
|
||||
# 9a: Phase 0c (added 2026-07-29) derives the required pin set from
|
||||
# gen/**.lean, so an unpinned model file is caught before compilation.
|
||||
# 9b: with the rogue file pinned — i.e. an author who added it
|
||||
# deliberately — the dead-file gate in Phase 2 must still catch that it is
|
||||
# absent from the compile manifest.
|
||||
printf '/- rogue -/\ntheorem rogue_gen : 1 = 1 := rfl\n' > "$T/gen/LTLAcc/Rogue.lean"
|
||||
if SKIP_FIDELITY=1 "$T/check.sh" > "$T/check9.out" 2>&1; then
|
||||
echo " ✗ case 9: check.sh PASSED with unmanifested gen/LTLAcc/Rogue.lean"; exit 1
|
||||
if SKIP_FIDELITY=1 "$T/check.sh" > "$T/check9a.out" 2>&1; then
|
||||
echo " ✗ case 9a: check.sh PASSED with an unpinned gen/LTLAcc/Rogue.lean"; exit 1
|
||||
fi
|
||||
grep -q "DEAD FILE (gen): gen/LTLAcc/Rogue.lean" "$T/check9.out" || {
|
||||
echo " ✗ case 9: check.sh failed without DEAD FILE (gen) diagnosis"; tail -5 "$T/check9.out"; exit 1; }
|
||||
echo " ✓ case 9 unmanifested gen module: check.sh dies with DEAD FILE (gen)"
|
||||
grep -q "does not match HARNESS.sha256" "$T/check9a.out" || {
|
||||
echo " ✗ case 9a: check.sh failed without the harness-set diagnosis"; tail -5 "$T/check9a.out"; exit 1; }
|
||||
echo " ✓ case 9a unpinned gen module: Phase 0c dies with a harness-set mismatch"
|
||||
|
||||
( cd "$T" && sha256sum gen/LTLAcc/Rogue.lean >> HARNESS.sha256 )
|
||||
if SKIP_FIDELITY=1 "$T/check.sh" > "$T/check9b.out" 2>&1; then
|
||||
echo " ✗ case 9b: check.sh PASSED with unmanifested gen/LTLAcc/Rogue.lean"; exit 1
|
||||
fi
|
||||
grep -q "DEAD FILE (gen): gen/LTLAcc/Rogue.lean" "$T/check9b.out" || {
|
||||
echo " ✗ case 9b: check.sh failed without DEAD FILE (gen) diagnosis"; tail -5 "$T/check9b.out"; exit 1; }
|
||||
echo " ✓ case 9b pinned but unmanifested: check.sh dies with DEAD FILE (gen)"
|
||||
|
||||
rm -rf "$WORK"
|
||||
trap - ERR
|
||||
echo "=== SELF-TEST GREEN: 9 attack cases defeated + positive control ==="
|
||||
echo "=== SELF-TEST GREEN: 10 attack cases defeated + positive control ==="
|
||||
|
|
|
|||
Loading…
Reference in a new issue