From ee57493f262b48d5d9a04b917f4e1985b8d0657e Mon Sep 17 00:00:00 2001 From: mrwulf Date: Fri, 31 Jul 2026 17:53:31 +0200 Subject: [PATCH] P2-c: classify and pin the extraction boundary MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Aeneas emits a *_Template.lean naming everything the extracted code needs from outside itself — the extraction's own statement of its boundary. extract.sh has always said, in prose, "after regenerating, diff the template against the hand-written file". Prose is not a gate, and the diff cannot be one: the two files legitimately differ in almost every line, holes and Aeneas comments against real definitions and modeling policy. MEASURING FIRST CHANGED WHAT THIS ITEM SHOULD BE. The TODO offered two options — enforce the diff, or pin both files — and the answer turned out to be neither. Both files were ALREADY byte-pinned by Phase 0b. And two further things stand here: the generated Funs.lean imports the model and CALLS these externals, so the Lean compiler enforces their TYPES wherever the extracted code uses them; and the per-certificate exact cones catch any external that becomes, or stops being, an assumption anything depends on. What none of those three sees is the CLASSIFICATION: for each name the extraction asks for, whether this repository answers with an ASSUMPTION or with a PROOF. That is the tier-A/B claim the documents make in prose — the curve calls and the three curve types resolve to proven definitions rather than axioms, because gen/CurveField/Funs.lean opens `namespace curve25519_dalek` and so defines the very names Aeneas asks for. Nothing checked it. A regeneration that renamed one, or a model that quietly answered one with an axiom instead, would have left the documents claiming a proof where the repository had an assumption. Phase 0d recomputes the classification with model-correspondence.py (namespace-aware, so a definition inside a namespace counts under its full name) and requires equality with the committed MODEL-CORRESPONDENCE.txt. UNRESOLVED — the extraction asking for something nothing here provides — is a hard failure. dalek 43 MODEL 8 PROVEN 3 EXTRA anza 38 MODEL 0 PROVEN 4 EXTRA (no CurveSig crate) risc0 36 MODEL 8 PROVEN 4 EXTRA betrusted 35 MODEL 8 PROVEN 4 EXTRA selftest-correspondence.sh, five cases, negative-tested by disabling the comparison. The case that matters is 2: a PROVEN external answered by an axiom instead. No name changes anywhere, every byte pin still matches, and it compiles, because the signature is unchanged — before Phase 0d nothing in the button could tell. Trap recorded for whoever extends it: case 3 first deleted the PROVEN rows, which was VACUOUS on anza, since anza has none — it removed nothing, the table still matched, and the case passed while testing nothing. It now deletes the first row whatever its verdict AND asserts the file changed. extract.sh now points at the gate instead of asking a human to look. Certified by a full sweep: both buttons, all four forks, purged trees, machine otherwise idle. 8/8 green. --- TRUSTED-BASE.md | 14 +++ verification/HARNESS.sha256 | 7 +- verification/MODEL-CORRESPONDENCE.txt | 43 ++++++++ verification/check.sh | 58 +++++++++++ verification/extract.sh | 10 ++ verification/model-correspondence.py | 74 ++++++++++++++ verification/selftest-correspondence.sh | 126 ++++++++++++++++++++++++ 7 files changed, 330 insertions(+), 2 deletions(-) create mode 100644 verification/MODEL-CORRESPONDENCE.txt create mode 100755 verification/model-correspondence.py create mode 100755 verification/selftest-correspondence.sh diff --git a/TRUSTED-BASE.md b/TRUSTED-BASE.md index 44a4504..0261497 100644 --- a/TRUSTED-BASE.md +++ b/TRUSTED-BASE.md @@ -46,6 +46,20 @@ running Rust code. Everything else is machine-checked. rejected, and are. Before those cases existed nothing in the harness distinguished "this tier needs no hash oracle" from "this tier happens not to use one today". + **What answers each external, and whether it is a proof or an assumption.** + Aeneas emits a `*_Template.lean` naming everything the extracted code needs + from outside itself — the extraction's own statement of its boundary. Phase + 0d requires every one of those names to be answered either by the + hand-written model beside it (an assumption, then governed by the axiom gate + and the cones) or by a real definition already in the extracted corpus, and + requires the classification to equal the committed + `MODEL-CORRESPONDENCE.txt` exactly. That second class is the tier-A/B claim + this document makes above — the curve calls and curve types resolving to + proven definitions rather than to axioms — and until 2026-07-31 it was prose + that nothing checked. `selftest-correspondence.sh` attacks it, including the + case that matters most: a PROVEN external answered by an axiom instead, + which changes no name anywhere, leaves every byte pin matching, and compiles + cleanly because the signature is unchanged. 6. **Compilation of Rust to machine code** (rustc backend) is out of scope, as is side-channel behaviour (timing, speculation). The proofs are about functional correctness at the MIR/LLBC level. diff --git a/verification/HARNESS.sha256 b/verification/HARNESS.sha256 index 17ecdfd..902786a 100644 --- a/verification/HARNESS.sha256 +++ b/verification/HARNESS.sha256 @@ -1,12 +1,14 @@ 12dc724bffd590e6f706573d97bf07425b8f268a1be2d72a3bbd9aef48f9c277 AUDIT-MANIFEST.txt 6b25b7e261633f4fa703de0be1dfd3c263d043107e3b3225d4c5079a93b6a64d check-scalar.sh -1bd3caaa12b1db17dd2d625410fdbf81b6ba3387b819da82b4e621e6477696b4 check.sh -64602c31ef34e740a0a27431534fa2ca65b6cd90a99cb17e119915a82a38b474 extract.sh +9f2e324cc6569872aea18437f87ec0f1ea715a230e9fec42ea3430ca084ba3f3 check.sh +fdd1337f1f44fa73fdb14e6cda001e4e7358513b9c178c517456b24511d96a8e extract.sh 52afbe130c5551686f45643a35065729fd5bb8166b5fa3db67b74c60ba3eff62 GEN-MODEL.sha256 6033c86eb08b4c2ea0bd7cdbd2cfb5748059179ece3efa9673270dc17a2e38b9 inventory-allowlist-scalar.txt b2c79ccf7005051946d49ee1fb0b51fbee58819c5909c6efb00965547b56fa5d inventory-allowlist.txt 0bb01bc4abaafa8537d460682004d1f336980b28bc4fe1968bcc9c3bc3bc71ba inventory_gate.sh 736ea4be712e1b5bcda10ecb466f0dec7008a2a36eabdfd77563976299c43cce lean-guard +fde2e987a9f69cb9f39b18ab8b405d73db4d9abee9f714f8be993a40ef617c03 model-correspondence.py +a8e2fa50e6f14278e9d088eb6339debc846419df297b0cdb2b9fbb77294545bd MODEL-CORRESPONDENCE.txt 34c102ca7a38719ef8890b8dedd3f07c3f92e10ae4a1ce2c47a7db4d53523354 Proofs/Audit.lean 4f23dc814f02d3d0355327fd01ab9092dae8b87c76cc296058693428f75e9c7e Proofs/InventoryBasic.lean 84bc670991fd7456d8c8569ff7b7c32410513a63d3cb7fe8877bb19a82d36a7d Proofs/InventoryCore.lean @@ -14,6 +16,7 @@ b2c79ccf7005051946d49ee1fb0b51fbee58819c5909c6efb00965547b56fa5d inventory-allo 6fbeb50d3951c7c5ac593f6dec91096fc798123ed0c500fdfa39fb20b118ea78 Proofs/InventoryScalar.lean bf71e8d4eb312ebc687bf7e218d90b910543cd92e782078174868d012aca7250 selftest-auditonly.sh eb81df6d154b413b243ad282e3b1bfec92fde158a215f89993c08586a121f171 selftest-axgate.sh +73d629325b667e715c7e6a633332950ace019e18e6d143ec5cb25846b8ddeb80 selftest-correspondence.sh 3d5898161d663eccad162269a5a6c102319077e22e1f2d89a8bfcab6926d29f6 selftest-harness.sh 1df031a075fc438c5229d01cbc44ee6ac489a272cd736624f7ca45ef1a4ddb7f selftest-inventory.sh 26f10a749e03cecd7ad173d0d621498386444d8e347f606a99a2fadb06738d86 selftest-shapes.sh diff --git a/verification/MODEL-CORRESPONDENCE.txt b/verification/MODEL-CORRESPONDENCE.txt new file mode 100644 index 0000000..a5d6608 --- /dev/null +++ b/verification/MODEL-CORRESPONDENCE.txt @@ -0,0 +1,43 @@ +CurveField/FunsExternal|Array.Insts.CoreHashHash.hash|MODEL +CurveField/FunsExternal|Bool.Insts.CoreConvertFromChoice.from|MODEL +CurveField/FunsExternal|Slice.Insts.CoreFmtDebug.fmt|MODEL +CurveField/FunsExternal|Slice.Insts.SubtleConstantTimeEq.ct_eq|MODEL +CurveField/FunsExternal|U32.Insts.CoreIterRangeStep.backward_checked|MODEL +CurveField/FunsExternal|U32.Insts.CoreIterRangeStep.forward_checked|MODEL +CurveField/FunsExternal|U32.Insts.CoreIterRangeStep.steps_between|MODEL +CurveField/FunsExternal|U64.Insts.SubtleConditionallySelectable.conditional_assign|MODEL +CurveField/FunsExternal|U64.Insts.SubtleConditionallySelectable.conditional_select|MODEL +CurveField/FunsExternal|U64.Insts.SubtleConditionallySelectable.conditional_swap|MODEL +CurveField/FunsExternal|U8.Insts.CoreHashHash.hash|MODEL +CurveField/FunsExternal|U8.Insts.SubtleConstantTimeEq.ct_eq|MODEL +CurveField/FunsExternal|backend.scalar_fits_in_128_bits|MODEL +CurveField/FunsExternal|backend.serial.scalar_mul.variable_base.mul|MODEL +CurveField/FunsExternal|backend.serial.scalar_mul.vartime_triple_base.mul_128_128_256_prechecked|MODEL +CurveField/FunsExternal|core.fmt.Formatter.debug_struct_field2_finish|MODEL +CurveField/FunsExternal|core.ops.range.RangeFull.Insts.CoreSliceIndexSliceIndexSliceSlice.get|MODEL +CurveField/FunsExternal|core.ops.range.RangeFull.Insts.CoreSliceIndexSliceIndexSliceSlice.get_mut|MODEL +CurveField/FunsExternal|core.ops.range.RangeFull.Insts.CoreSliceIndexSliceIndexSliceSlice.index|MODEL +CurveField/FunsExternal|core.result.Result.Insts.CoreOpsTry_traitTry.branch|MODEL +CurveField/FunsExternal|ed25519.Signature.r_bytes|MODEL +CurveField/FunsExternal|ed25519.Signature.s_bytes|MODEL +CurveField/FunsExternal|ed_sigs.sha512_hash3|MODEL +CurveField/FunsExternal|edwards.CompressedEdwardsY.Insts.CoreCmpEq.assert_fields_are_eq|MODEL +CurveField/FunsExternal|edwards.CompressedEdwardsY.from_slice|MODEL +CurveField/FunsExternal|edwards.EdwardsPoint.Insts.CoreCmpEq.assert_fields_are_eq|MODEL +CurveField/FunsExternal|edwards.EdwardsPoint.Insts.CoreIterTraitsAccumSum.sum|MODEL +CurveField/FunsExternal|edwards.EdwardsPoint.Insts.SubtleConditionallySelectable.conditional_swap|MODEL +CurveField/FunsExternal|edwards.affine.AffinePoint.Insts.CoreCmpEq.assert_fields_are_eq|MODEL +CurveField/FunsExternal|field.FieldElement51.internal_invert_batch|MODEL +CurveField/FunsExternal|subtle.Choice.Insts.CoreConvertFromU8.from|MODEL +CurveField/FunsExternal|subtle.Choice.Insts.CoreOpsBitBitAndChoiceChoice.bitand|MODEL +CurveField/FunsExternal|subtle.Choice.Insts.CoreOpsBitBitOrChoiceChoice.bitor|MODEL +CurveField/FunsExternal|subtle.Choice.unwrap_u8|MODEL +CurveField/FunsExternal|subtle.ConditionallySelectable.conditional_assign.default|MODEL +CurveField/FunsExternal|subtle.ConditionallySelectable.conditional_swap.default|MODEL +CurveField/FunsExternal|backend.vector.scalar_mul.variable_base.spec_avx2.mul|EXTRA +CurveField/FunsExternal|backend.vector.scalar_mul.vartime_double_base.spec_avx2.mul|EXTRA +CurveField/FunsExternal|core.result.Result.Insts.CoreOpsTry_traitFromResidualResultInfallibleE.from_residual|EXTRA +CurveField/FunsExternal|hash|EXTRA +CurveField/TypesExternal|ed25519.Signature|MODEL +CurveField/TypesExternal|subtle.Choice|MODEL +CORRESPONDENCE-COUNT|42 diff --git a/verification/check.sh b/verification/check.sh index cae6c48..4276cc7 100755 --- a/verification/check.sh +++ b/verification/check.sh @@ -318,6 +318,7 @@ echo " $(wc -l < "$HERE/GEN-MODEL.sha256") extracted-model files match their pi HARNESS_EXTRA=( AUDIT-MANIFEST.txt # the statement block Phase 3c's digest is taken over GEN-MODEL.sha256 # the extracted-model pins Phase 0b enforces + MODEL-CORRESPONDENCE.txt # the extraction boundary Phase 0d recomputes inventory-allowlist.txt # the audit surface Phase 2c diffs against inventory-allowlist-scalar.txt # the scalar layer's audit surface (second button) Proofs/Audit.lean # the audit driver: it computes the digest it is judged by @@ -351,6 +352,63 @@ if ! ( cd "$HERE" && sha256sum -c --quiet HARNESS.sha256 ) ; then exit 1 fi echo " $(wc -l < "$HERE/HARNESS.sha256") harness files match their pins" +# ── Phase 0d: template/model correspondence ───────────────────────────────── +# WHAT AENEAS'S TEMPLATE IS. When Aeneas extracts the Rust it also emits, for +# each crate, a *_Template.lean naming everything the extracted code needs from +# OUTSIDE itself. That template is the extraction's own statement of its +# boundary. The hand-written *External.lean beside it is our answer to that +# statement, and `extract.sh` has always said, in prose, "after regenerating, +# diff the template against the hand-written file". Prose is not a gate. +# +# WHAT THIS ADDS, given that three other things already stand here. Phase 0b +# byte-pins both files, so neither can drift from its pin unnoticed. The +# generated Funs.lean imports the model and CALLS these externals, so the Lean +# compiler already enforces their types wherever the extracted code uses them. +# The per-certificate exact cones catch any external that becomes — or stops +# being — an assumption anything depends on. What none of those three sees is +# the CLASSIFICATION: for each name the extraction asks for, whether this +# repository answers with an assumption or with a proof. +# +# That distinction is the tier-A/B claim, and it was prose until 2026-07-31. +# The docs say the curve calls (compress, as_bytes, +# vartime_double_scalar_mul_basepoint, from_bytes_mod_order{,_wide}) and the +# three curve TYPES resolve to the PROVEN model's own definitions rather than +# to axioms — because gen/CurveField/Funs.lean opens `namespace +# curve25519_dalek`, so the names Aeneas asks for are the names it defines. +# Nothing checked it. A regeneration that renamed one of those, or a model that +# quietly answered one with an axiom instead, would have left the documents +# claiming a proof where the repository now had an assumption. +# +# model-correspondence.py recomputes the classification from the files — +# namespace-aware, so a definition inside `namespace curve25519_dalek` counts +# under its full name — and the result must equal the committed table exactly. +# UNRESOLVED is a hard failure in the tool itself: the extraction asking for +# something this repository does not provide at all. +echo "=== Phase 0d: template/model correspondence ===" +CORR_FILE="$HERE/MODEL-CORRESPONDENCE.txt" +if [ ! -s "$CORR_FILE" ]; then + echo "FATAL: MODEL-CORRESPONDENCE.txt is missing or empty — the extraction boundary is unpinned." + exit 1 +fi +CORR_OBSERVED=$(cd "$HERE" && python3 model-correspondence.py .) || { + echo "$CORR_OBSERVED" | grep UNRESOLVED | sed 's/^/ /' + echo "MODEL CORRESPONDENCE FAILED: the extraction declares an external that neither" + echo "the hand-written model nor the proven corpus provides." + exit 1 +} +if ! diff -u "$CORR_FILE" <(printf '%s\n' "$CORR_OBSERVED") > /tmp/corr-diff.$$ 2>&1; then + echo " MODEL CORRESPONDENCE DRIFT (< committed, > observed):" + sed -n '4,24p' /tmp/corr-diff.$$ | sed 's/^/ /' + rm -f /tmp/corr-diff.$$ + echo "MODEL CORRESPONDENCE FAILED: an external changed how it is answered." + exit 1 +fi +rm -f /tmp/corr-diff.$$ +echo " $(grep -c '|MODEL$' "$CORR_FILE") externals answered by the hand-written model (assumptions)" +echo " $(grep -c '|PROVEN$' "$CORR_FILE") answered by PROVEN definitions in the extracted corpus" +echo " $(grep -c '|EXTRA$' "$CORR_FILE") model declarations beyond what the extraction asks for" +echo "" + # ── Phase 1: stub + axiom-smuggling audit ─────────────────────────────────── echo "=== Phase 1: stub audit ===" if grep -rn 'by trivial' "$HERE"/Proofs/*Spec*.lean 2>/dev/null; then diff --git a/verification/extract.sh b/verification/extract.sh index 67e08d0..70ded29 100755 --- a/verification/extract.sh +++ b/verification/extract.sh @@ -18,6 +18,16 @@ # After regenerating, diff the templates against the hand-written files: # diff gen/CurveField/FunsExternal_Template.lean gen/CurveField/FunsExternal.lean # +# That diff is a READING aid, not a gate — the two files legitimately differ in +# almost every line (the template holds holes and Aeneas's own comments; the +# model holds real definitions and the modeling policy). What IS enforced, by +# check.sh Phase 0d, is the classification: every name the template declares +# must be answered either by the hand-written model or by a real definition in +# the proven corpus, and which of the two must match MODEL-CORRESPONDENCE.txt. +# Regenerate that table with `python3 model-correspondence.py .` and commit the +# change deliberately — a proof silently becoming an assumption is exactly what +# the phase exists to stop. +# # Usage: ./extract.sh set -euo pipefail diff --git a/verification/model-correspondence.py b/verification/model-correspondence.py new file mode 100755 index 0000000..1563013 --- /dev/null +++ b/verification/model-correspondence.py @@ -0,0 +1,74 @@ +#!/usr/bin/env python3 +"""Classify every external the extraction declares. + +For each gen//_Template.lean, Aeneas states what the extracted Rust +needs from outside. Each such name must be provided by exactly one of: + + MODEL — declared in the hand-written sibling gen//.lean: an + assumption, which the axiom gate and the per-certificate cones + then govern; + PROVEN — resolved to a real definition in the proven corpus, because a + module of this repository declares it (namespace-aware). This is + the valuable case and the one the docs claim for the tier-A/B + curve calls; nothing has ever checked it. + +Anything else is drift: the extraction asks for something this repository does +not provide. +""" +import re, sys, os, glob + +DECL = re.compile( + r'^[ \t]*(?:@\[[^\]]*\]\s*)?(?:private |protected |noncomputable |unsafe )*' + r'(axiom|def|abbrev|opaque|structure|inductive)[ \t]+([A-Za-z_][A-Za-z0-9_.\'!?]*)') +NS = re.compile(r'^[ \t]*(namespace|end)[ \t]+([A-Za-z_][A-Za-z0-9_.\']*)') + +def declared(path): + """Fully-qualified names declared in one file, honouring namespaces.""" + names, stack = set(), [] + for line in open(path, encoding='utf-8', errors='replace'): + m = NS.match(line) + if m: + if m.group(1) == 'namespace': + stack.append(m.group(2)) + elif stack and stack[-1] == m.group(2): + stack.pop() + continue + d = DECL.match(line) + if d: + names.add('.'.join(stack + [d.group(2)]) if stack else d.group(2)) + return names + +def main(root): + gen = os.path.join(root, 'gen') + templates = sorted(glob.glob(os.path.join(gen, '*', '*_Template.lean'))) + # The proven corpus: every generated module that is neither a template nor + # a hand-written model. These are the files Aeneas produced from Rust. + models = {t.replace('_Template', '') for t in templates} + corpus = set() + for f in sorted(glob.glob(os.path.join(gen, '*', '*.lean'))): + if f in models or f.endswith('_Template.lean'): + continue + corpus |= declared(f) + + rows, unresolved = [], [] + for t in templates: + model = t.replace('_Template', '') + rel = os.path.relpath(t, gen).replace('_Template.lean', '') + tnames = declared(t) + mnames = declared(model) if os.path.exists(model) else set() + for n in sorted(tnames): + if n in mnames: + rows.append(f'{rel}|{n}|MODEL') + elif n in corpus: + rows.append(f'{rel}|{n}|PROVEN') + else: + rows.append(f'{rel}|{n}|UNRESOLVED') + unresolved.append(f'{rel}|{n}') + for n in sorted(mnames - tnames): + rows.append(f'{rel}|{n}|EXTRA') + print('\n'.join(rows)) + print(f'CORRESPONDENCE-COUNT|{len(rows)}') + return 1 if unresolved else 0 + +if __name__ == '__main__': + sys.exit(main(sys.argv[1])) diff --git a/verification/selftest-correspondence.sh b/verification/selftest-correspondence.sh new file mode 100755 index 0000000..7fe9813 --- /dev/null +++ b/verification/selftest-correspondence.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +# ───────────────────────────────────────────────────────────────────────────── +# selftest-correspondence.sh — attacks check.sh Phase 0d. +# +# Phase 0d asserts HOW each external the extraction asks for is answered: with +# an assumption in the hand-written model, or with a proof already in the +# extracted corpus. The second class is the one the documents make a claim +# about — the curve calls and curve types are said to resolve to the proven +# model's own definitions rather than to axioms — and that claim was prose +# until this phase existed. +# +# 0 control: the committed table matches the files +# 1 the extraction asks for something NOTHING provides -> UNRESOLVED +# 2 a PROVEN external answered by an axiom in the model instead. This is the +# attack that matters: a proof silently downgraded to an assumption, in a +# name whose spelling does not change anywhere else. +# 3 a row deleted from the committed table -> drift +# 4 a row's verdict edited in the committed table -> drift +# +# No Lean: Phase 0d is pure text over gen/. Seconds, not minutes. +# ───────────────────────────────────────────────────────────────────────────── +set -uo pipefail +HERE="$(cd "$(dirname "$0")" && pwd)" +FAILURES=0 +STASH="$(mktemp -d)" + +cleanup() { + [ -f "$STASH/corr" ] && cp "$STASH/corr" "$HERE/MODEL-CORRESPONDENCE.txt" + [ -f "$STASH/model" ] && cp "$STASH/model" "$HERE/$MODEL_REL" + [ -f "$STASH/tmpl" ] && cp "$STASH/tmpl" "$HERE/$TMPL_REL" + rm -rf "$STASH" +} + +# Derive the victims from this repo rather than naming them: the forks do not +# share a gen/ layout (anza has no CurveSig crate at all, so it has no PROVEN +# rows and case 2 does not apply there). +TMPL_REL=$(cd "$HERE" && ls gen/*/FunsExternal_Template.lean | head -1) +MODEL_REL="${TMPL_REL/_Template/}" +PROVEN_ROW=$(grep -m1 '|PROVEN$' "$HERE/MODEL-CORRESPONDENCE.txt" || true) + +trap cleanup EXIT INT TERM +cp "$HERE/MODEL-CORRESPONDENCE.txt" "$STASH/corr" +cp "$HERE/$MODEL_REL" "$STASH/model" +cp "$HERE/$TMPL_REL" "$STASH/tmpl" + +# Phase 0d lifted from the shipping button. +{ echo 'set -euo pipefail' + echo "HERE=\"$HERE\"" + awk '/^# ── Phase 0d/{f=1} f&&/^# ── (Phase 1|Phases end)/{exit} f{print}' "$HERE/check.sh" +} > "$STASH/p0d.sh" +for want in 'Phase 0d' 'MODEL CORRESPONDENCE' 'model-correspondence.py'; do + grep -qF "$want" "$STASH/p0d.sh" || { + echo "FATAL: the lifted driver has no '$want' — check.sh's phase markers moved."; exit 1; } +done + +expect() { # expect