ltl-accumulator-verified/verification/selftest-harness.sh
mrwulf be4f742b29 verification: guard the accumulator's two lifts (port lift-guard, assert on the payload)
The four forks ship lift-guard.sh and call it from eight files each. This
repository had no lift-guard.sh at all and called it zero times, while lifting
Phase 0c (selftest-harness.sh) and Phase 3d (selftest_statements.sh). Its lifts
got a line-count sanity check and nothing else — no analysis of which variables
the payload READS against which the driver DEFINES, and after yesterday's fix
no errexit check either. It was the one place that class could still recur.

A SECOND defect surfaced while porting, and it is the more interesting one.
Both lifts assembled the driver inline, with no payload file anywhere:

    { echo 'set -euo pipefail'; echo "HERE=..."; sed -n '...' check.sh } > DRIVER
    if [ "$(wc -l < "$DRIVER")" -lt 40 ]; then ... refuse ...

so the size check measured the CONCATENATED DRIVER — bindings included. A lift
that recovered almost nothing still cleared the threshold on the strength of
the lines the test itself prepended. That is exactly the defect round-7
lift-guard-payload-assert reported and that the forks closed with the words
"all six lift sites now assert on payload.sh". Six counted the forks. These two
were never in the count, so a finding recorded as closed was measuring the
wrong population — the same error this estate keeps finding in new costumes.
Both sites now write payload.sh / payload3d.sh and assert on those.

lift-guard.sh is byte-identical to the forks' (4f61c113). It is executable, so
this repository's own self-deriving Phase 0c rule required a pin before it
would run at all: required set == pinned set, verified.

WIRING PROVEN RATHER THAN ASSUMED. With lift-guard.sh stubbed to exit 1, each
self-test halts AT the guard, naming its phase, before a single attack case
runs. Guard restored, pins re-verified.

Certified: button + all three self-tests GREEN (check.sh 86s, selftest_audit
258s, selftest-harness 1s, selftest_statements 34s). The other five
repositories are unchanged from the tree round-15 certified 50/50 and were not
re-run; this commit does not claim they were.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-08-04 13:39:19 +02:00

119 lines
6 KiB
Bash
Executable file

#!/usr/bin/env bash
# ─────────────────────────────────────────────────────────────────────────────
# selftest-harness.sh — adversarial self-test for check.sh Phase 0c.
#
# Phase 0c pins the scripts and policy files the button itself runs on. The
# attack it exists to stop is the cheapest one in the estate: don't touch the
# proofs at all, edit the checker. Round-5 review of the companion SLH-DSA
# repository stubbed the compiler wrapper alone and got ALL GREEN in 3.6
# seconds over deliberately destroyed proofs.
#
# Cases, each asserting a SPECIFIC diagnostic:
# 0 positive control: untouched tree passes
# 1 a pinned harness file edited by one byte → does not match its pin
# 2 a NEW executable appears, unpinned → set mismatch
# 3 an entry DELETED from HARNESS.sha256 → set mismatch, NOT a
# silent un-pin (this is the shape of the defect SLH-DSA round-6 found:
# dropping a key un-pinned two files with no diagnostic at all)
# 4 HARNESS.sha256 itself removed → fail-closed
#
# Phase 0c is lifted out of check.sh at run time, so the tested logic IS the
# shipping logic. Cheap: no Lean, runs in about a second.
# ─────────────────────────────────────────────────────────────────────────────
set -uo pipefail
HERE="$(cd "$(dirname "$0")" && pwd)"
FAILURES=0
STASH="$(mktemp -d)"
NEWEXE="$HERE/zz-selftest-helper.sh"
cleanup() {
[ -f "$STASH/HARNESS.sha256" ] && cp "$STASH/HARNESS.sha256" "$HERE/HARNESS.sha256"
[ -f "$STASH/victim" ] && cp "$STASH/victim" "$HERE/$VICTIM"
rm -f "$NEWEXE"
rm -rf "$STASH"
}
trap cleanup EXIT INT TERM
cp "$HERE/HARNESS.sha256" "$STASH/HARNESS.sha256"
# Lift Phase 0c. The two repo families end the phase differently, so accept
# either terminator rather than hardcoding one and silently lifting nothing.
# The payload is written to its OWN file before the driver is assembled, for two
# reasons that both bit this repository. The size sanity check below has to
# measure what was LIFTED; measured on the concatenated driver it counts the
# prepended bindings too, so a lift that recovered almost nothing still clears
# the threshold (round-7 lift-guard-payload-assert, closed in the four forks
# with "all six lift sites" — six counted the forks, and these two accumulator
# lifts were never in that count). And lift-guard needs payload and driver
# separately to tell what the phase READS from what the driver DEFINES.
DRIVER="$STASH/phase0c.sh"
PAYLOAD="$STASH/payload.sh"
awk '/^# ── Phase 0c/{f=1} f{print} /^# ── Phase 1|^echo "=== Phase 1/{if(f && !/Phase 0c/) exit}' "$HERE/check.sh" \
| sed '/^# ── Phase 1/d; /^echo "=== Phase 1/d' > "$PAYLOAD"
{
echo 'set -euo pipefail' # -e matches the button; see lift-drivers-drop-errexit
echo "HERE=\"$HERE\""
cat "$PAYLOAD"
} > "$DRIVER"
if [ "$(grep -c . "$PAYLOAD")" -lt 20 ]; then
echo "FATAL: could not lift Phase 0c out of check.sh — the phase markers moved."
echo "This self-test must attack the shipping gate; refusing to run against nothing."
exit 1
fi
"$HERE/lift-guard.sh" "$PAYLOAD" "$DRIVER" "check.sh Phase 0c" || exit 1
expect() { # expect <label> <want-rc> <want-substring>
local label="$1" want_rc="$2" want_txt="$3" out rc
out=$(bash "$DRIVER" 2>&1); rc=$?
if [ "$rc" -ne "$want_rc" ]; then
echo "$label: exit $rc, expected $want_rc"; echo "$out" | sed 's/^/ /'
FAILURES=$((FAILURES+1)); return
fi
if ! grep -qF "$want_txt" <<<"$out"; then
echo "$label: exit code right, diagnostic wrong (rejected for the wrong reason)"
echo " wanted: $want_txt"; echo "$out" | sed 's/^/ /'
FAILURES=$((FAILURES+1)); return
fi
echo "$label"
}
echo "=== selftest-harness: attacking check.sh Phase 0c ==="
# ── 0. positive control ────────────────────────────────────────────────────
expect "case 0 control: untouched harness passes" 0 "match their pins"
# ── 1. edit a pinned file. lean-guard is the pointed choice: it is the memory
# cap protecting this machine, and stubbing it is the demonstrated
# 3.6-second path to a false green.
VICTIM=lean-guard
cp "$HERE/$VICTIM" "$STASH/victim"
printf '\n# selftest\n' >> "$HERE/$VICTIM"
expect "case 1: edited lean-guard caught" 1 "does not match its pin"
cp "$STASH/victim" "$HERE/$VICTIM"
# ── 2. a new executable the button could shell out to ──────────────────────
printf '#!/bin/sh\necho "unpinned"\n' > "$NEWEXE"; chmod +x "$NEWEXE"
expect "case 2: new unpinned executable caught" 1 "does not match HARNESS.sha256"
rm -f "$NEWEXE"
# ── 3. delete a pin entry: the set must be derived from the filesystem, not
# read out of the map being consulted, or this is a silent un-pin.
grep -v " ${VICTIM}\$" "$STASH/HARNESS.sha256" > "$HERE/HARNESS.sha256"
expect "case 3: deleted pin entry is a failure, not a silent un-pin" 1 "does not match HARNESS.sha256"
cp "$STASH/HARNESS.sha256" "$HERE/HARNESS.sha256"
# ── 4. absence must not pass for cleanliness ───────────────────────────────
rm -f "$HERE/HARNESS.sha256"
expect "case 4: missing pin file is fail-closed" 1 "the harness is unpinned"
cp "$STASH/HARNESS.sha256" "$HERE/HARNESS.sha256"
# ── 5. restored ────────────────────────────────────────────────────────────
expect "case 5: restored to green" 0 "match their pins"
echo ""
if [ "$FAILURES" -eq 0 ]; then
echo "SELFTEST PASSED — Phase 0c rejects harness tampering for the stated reason."
exit 0
fi
echo "SELFTEST FAILED: $FAILURES case(s) did not behave as claimed."
exit 1