mirror of
https://github.com/saymrwulf/proof-aware-crypto-tooling-agent.git
synced 2026-09-03 19:53:43 +00:00
mrwulf
2587cc018b
REAL DEFECT found by the operator-ordered doc-freshness audit:
published_assets.py still carried the PRE-HARDENING fail-open verify.py
and the pre-Tier-2 README as the templates that log-publish drops into
the mirror — the next publish would have silently overwritten the
round-13-hardened fail-closed verifier and the corrected README with
the old versions. Fixed:
- published_assets.py regenerated from the canonical mirror files
(byte-identity verified by round-trip exec), now also carrying
verify_selftest.py; SYNC RULE documented in the module docstring.
- transparency_log.publish() now writes verify_selftest.py too.
- NEW tests/test_published_assets.py pins the security-critical markers
(fail-closed FATAL, RECEIPT_TYPE, verify_receipt, --all receipt
coverage, required fingerprint) so template drift fails CI instead of
shipping.
- test_web_and_witness updated to the hardened verifier's markers —
the published test log now passes FULL signature mode end to end
('RESULT: OK [full]'), a stronger assertion than the old string.
Doc refresh in the same pass:
- llms.txt: thirteen leaves + entry-13 self-attestation + fail-closed
verifier; paper line -> new title, 23 pages, v0.2/v0.1 archives.
- Course (generator + generated 06b notebook): 'the git hash IS the
content hash' -> 'the commit pins the exact source tree';
'irrevocably part of the log every other agent sees' -> 'committed to
the log's signed view, which any agent can compare' (the two Tier-2
scope corrections had never reached the teaching material).
- test_paper_verifiers.py docstring rescoped: its 164k counts are the
archived v0.2 report's citation; the current paper cites the corpus
harness and makes no extensional-equality claim.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
262 lines
12 KiB
Text
262 lines
12 KiB
Text
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Lecture 6b: THE AGENT'S SIDE - Verifying Inclusion (one of many)\n",
|
|
"\n",
|
|
"> **DOMAIN BANNER - read this first.** In this notebook YOU ARE AN\n",
|
|
"> AGENT. There are **many** of you. You possess exactly three\n",
|
|
"> things: the provider's public key, the evidence files, and about\n",
|
|
"> forty lines of hashing code. You do NOT possess Lean, a proof\n",
|
|
"> toolchain, or the provider's private key - and you never will\n",
|
|
"> need them. Everything below runs in milliseconds. If a cell in\n",
|
|
"> this notebook needed Lean, the design would have failed.\n",
|
|
"\n",
|
|
"This is **Merkle proof verification, not Lean verification** - the\n",
|
|
"agent checks WHERE a statement sits, never re-derives WHY it is true.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Learning Objectives\n",
|
|
"\n",
|
|
"- Implement the complete inclusion verifier from scratch - hashlib only, no pacta imports for the core.\n",
|
|
"- Verify a REAL receipt against the REAL signed tree head.\n",
|
|
"- See the inclusion path in the picture of the real 8-leaf log.\n",
|
|
"- Read the provider's self-check (\"dogfood in both directions\") from the signature block and say what it does and does not prove.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## The whole verifier, from scratch\n",
|
|
"\n",
|
|
"To make the cost asymmetry unmistakable, here is the ENTIRE core of\n",
|
|
"what an agent must implement - RFC 9162 inclusion verification in\n",
|
|
"~25 lines of standard-library Python. Read every line; this is all\n",
|
|
"the cryptographic machinery your trust rests on (plus one Ed25519\n",
|
|
"signature check):\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import hashlib\n",
|
|
"\n",
|
|
"def leaf_hash(data: bytes) -> bytes:\n",
|
|
" return hashlib.sha256(b\"\\x00\" + data).digest()\n",
|
|
"\n",
|
|
"def node_hash(left: bytes, right: bytes) -> bytes:\n",
|
|
" return hashlib.sha256(b\"\\x01\" + left + right).digest()\n",
|
|
"\n",
|
|
"def verify_inclusion(leaf: bytes, index: int, size: int, proof: list, root: bytes) -> bool:\n",
|
|
" if index >= size:\n",
|
|
" return False\n",
|
|
" fn, sn = index, size - 1\n",
|
|
" node = leaf_hash(leaf)\n",
|
|
" for sibling in proof:\n",
|
|
" if sn == 0:\n",
|
|
" return False\n",
|
|
" if fn % 2 == 1 or fn == sn:\n",
|
|
" node = node_hash(sibling, node)\n",
|
|
" if fn % 2 == 0:\n",
|
|
" while fn % 2 == 0 and fn != 0:\n",
|
|
" fn //= 2\n",
|
|
" sn //= 2\n",
|
|
" else:\n",
|
|
" node = node_hash(node, sibling)\n",
|
|
" fn //= 2\n",
|
|
" sn //= 2\n",
|
|
" return sn == 0 and node == root\n",
|
|
"\n",
|
|
"print(\"the agent's entire Merkle toolbox: 3 functions,\", \"no imports beyond hashlib\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Apply it to the REAL receipt\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"import json\n",
|
|
"from pathlib import Path\n",
|
|
"import sys\n",
|
|
"\n",
|
|
"repo_root = Path.cwd()\n",
|
|
"if not (repo_root / \"src\" / \"pacta\").exists():\n",
|
|
" repo_root = repo_root.parent\n",
|
|
"sys.path.insert(0, str(repo_root / \"src\"))\n",
|
|
"from pacta.yamlio import load_data\n",
|
|
"from pacta.signing import canonical_json\n",
|
|
"\n",
|
|
"att = load_data(repo_root / \"evidence\" / \"dalek-ed25519.attestation.yaml\")\n",
|
|
"receipt = load_data(repo_root / \"evidence\" / \"dalek-ed25519.receipt.yaml\")\n",
|
|
"\n",
|
|
"leaf_bytes = canonical_json({\"schema_version\": 1, \"type\": \"pacta.transparency.attestation_leaf.v1\", \"attestation\": att})\n",
|
|
"proof = [bytes.fromhex(h) for h in receipt[\"inclusion_proof\"]]\n",
|
|
"root = bytes.fromhex(receipt[\"sth\"][\"root_hash\"])\n",
|
|
"\n",
|
|
"ok = verify_inclusion(leaf_bytes, receipt[\"leaf_index\"], receipt[\"tree_size\"], proof, root)\n",
|
|
"print(f\"leaf {receipt['leaf_index']} of {receipt['tree_size']}, {len(proof)} siblings -> inclusion:\", ok)\n",
|
|
"assert ok\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## One signature check completes the chain\n",
|
|
"\n",
|
|
"Inclusion binds the attestation to a root; the signature binds the\n",
|
|
"root to the provider. Note what the AGENT learns from the signature\n",
|
|
"block's `signing_provenance`: the provider signed this root with the\n",
|
|
"merkleized library and Merkle-verified that library's own leaf first\n",
|
|
"- dogfood in both directions. The agent still re-checks inclusion\n",
|
|
"itself (above); the provenance is the provider's discipline made\n",
|
|
"visible, not a substitute for the agent's check.\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"from pacta.transparency import verify_signed_tree_head\n",
|
|
"from pacta.sthstore import check_sth_against_store\n",
|
|
"import tempfile\n",
|
|
"\n",
|
|
"ok, diagnostics, statuses = verify_signed_tree_head(receipt[\"sth\"], repo_root / \"evidence\" / \"provider.ed25519.pub\")\n",
|
|
"print(\"STH signature:\", statuses.get(\"ed25519\"), \"| verified on backend:\", statuses.get(\"ed25519_backend\"))\n",
|
|
"print(\"provider's own discipline, as recorded in the signature block:\")\n",
|
|
"print(json.dumps(receipt[\"sth\"][\"signatures\"][\"ed25519\"].get(\"signing_provenance\", {}), indent=1))\n",
|
|
"with tempfile.TemporaryDirectory() as tmp:\n",
|
|
" pin = check_sth_against_store(receipt[\"sth\"], Path(tmp) / \"pins.json\")\n",
|
|
" print(\"pin store:\", pin.action)\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## The picture: where your leaf sits in the real log\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": [
|
|
"# reconstruct the on-path node hashes from the receipt alone - the\n",
|
|
"# agent never needs the other leaves, only the siblings.\n",
|
|
"def svg_inclusion(receipt, width=980):\n",
|
|
" size, index = receipt[\"tree_size\"], receipt[\"leaf_index\"]\n",
|
|
" proof = receipt[\"inclusion_proof\"]\n",
|
|
" rows = [f'<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"{width}\" height=\"330\" font-family=\"monospace\" font-size=\"11\">']\n",
|
|
" rows.append(f'<rect x=\"4\" y=\"4\" width=\"{width-8}\" height=\"322\" fill=\"none\" stroke=\"#3b4d8f\" stroke-width=\"2\" rx=\"8\"/>')\n",
|
|
" rows.append('<text x=\"16\" y=\"24\" fill=\"#3b4d8f\" font-size=\"13\" font-weight=\"bold\">AGENT: verifies only the highlighted path - everything grey is somebody else's data</text>')\n",
|
|
" span = width / (size + 1)\n",
|
|
" for i in range(size):\n",
|
|
" x = span * (i + 1)\n",
|
|
" me = i == index\n",
|
|
" fill = \"#e2f2e9\" if me else \"#f4f4f6\"\n",
|
|
" stroke = \"#1e7f4f\" if me else \"#bbb\"\n",
|
|
" rows.append(f'<rect x=\"{x-42}\" y=\"250\" width=\"84\" height=\"30\" fill=\"{fill}\" stroke=\"{stroke}\" stroke-width=\"{2 if me else 1}\" rx=\"4\"/>')\n",
|
|
" label = \"YOUR leaf\" if me else f\"leaf {i}\"\n",
|
|
" rows.append(f'<text x=\"{x}\" y=\"269\" text-anchor=\"middle\" fill=\"{stroke}\">{label}</text>')\n",
|
|
" node = leaf_hash(leaf_bytes)\n",
|
|
" y = 250\n",
|
|
" x = span * (index + 1)\n",
|
|
" for depth, sib in enumerate(proof):\n",
|
|
" ny = y - 60\n",
|
|
" nx = x # visual simplification: path rises vertically\n",
|
|
" rows.append(f'<rect x=\"{nx-52}\" y=\"{ny-16}\" width=\"104\" height=\"30\" fill=\"#fdf0da\" stroke=\"#a86a10\" rx=\"4\"/>')\n",
|
|
" rows.append(f'<text x=\"{nx}\" y=\"{ny+3}\" text-anchor=\"middle\" fill=\"#a86a10\">sibling {depth}: {sib[:10]}\u2026</text>')\n",
|
|
" rows.append(f'<line x1=\"{x}\" y1=\"{y-16 if depth else 250}\" x2=\"{nx}\" y2=\"{ny+14}\" stroke=\"#a86a10\"/>')\n",
|
|
" y = ny\n",
|
|
" rows.append(f'<rect x=\"{x-135}\" y=\"{y-70}\" width=\"270\" height=\"30\" fill=\"#eef\" stroke=\"#3b4d8f\" rx=\"4\"/>')\n",
|
|
" rows.append(f'<text x=\"{x}\" y=\"{y-50}\" text-anchor=\"middle\" fill=\"#3b4d8f\">signed root {receipt[\"sth\"][\"root_hash\"][:14]}\u2026 (dogfood-signed)</text>')\n",
|
|
" rows.append(f'<line x1=\"{x}\" y1=\"{y-16}\" x2=\"{x}\" y2=\"{y-40}\" stroke=\"#3b4d8f\"/>')\n",
|
|
" rows.append(\"</svg>\")\n",
|
|
" return \"\".join(rows)\n",
|
|
"\n",
|
|
"svg = svg_inclusion(receipt)\n",
|
|
"try:\n",
|
|
" from IPython.display import SVG, display\n",
|
|
" display(SVG(svg))\n",
|
|
"except Exception:\n",
|
|
" print(\"(open in a notebook to render the figure)\")\n",
|
|
"print(f\"cost of everything in this notebook: ~{receipt['tree_size'].bit_length()} hashes + 1 signature check - milliseconds.\")\n"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Transports: files, git, and the online service\n",
|
|
"\n",
|
|
"Everything you verified above arrived as FILES - and that is a\n",
|
|
"feature: receipts are self-contained, so transport never carries\n",
|
|
"trust. The same log has two more faces. The GIT MIRROR\n",
|
|
"(`lean-transparency-log` on GitHub and on the provider's Forgejo)\n",
|
|
"publishes every leaf and every signed head - all cloners see the\n",
|
|
"same heads, which makes every cloner a WITNESS\n",
|
|
"(`pacta witness-audit` recomputes every prefix root from the\n",
|
|
"published leaves; run `python3 verify.py --all` in a clone for the\n",
|
|
"no-install version). The ONLINE SERVICE\n",
|
|
"(`ltl.zkdefi.org`) adds live endpoints: fetch\n",
|
|
"fresh evidence (`pacta log-fetch`), advance your pin with an\n",
|
|
"online consistency proof (`pacta sth-refresh`). The verification\n",
|
|
"you do afterwards is IDENTICAL in all three transports - this\n",
|
|
"notebook's ~25 lines never change.\n",
|
|
"\n",
|
|
"## Convinced - of what, exactly?\n",
|
|
"\n",
|
|
"After these cells pass, the agent knows: *the provider whose key I\n",
|
|
"pinned states that the Lean proofs of repository X at commit Y check\n",
|
|
"out with exactly the documented assumptions, and that statement is\n",
|
|
"committed to the log's signed view, which any agent can compare.* The agent then\n",
|
|
"clones commit Y (the commit pins the exact source tree) and builds it -\n",
|
|
"compiler and build remain declared trusted base until R5. Where a\n",
|
|
"claim lives (this notebook) and why it is true (the provider's Lean\n",
|
|
"replay, lecture 6a) never blur.\n",
|
|
"\n",
|
|
"## Exercises\n",
|
|
"\n",
|
|
"- Flip one byte of the leaf and re-run: which of the ~25 lines catches it?\n",
|
|
"- Flip one byte of the ROOT instead: what fails now, inclusion or the signature?\n",
|
|
"- Your entire verifier fits in one cell. List everything it does NOT check (freshness? consistency with your previous pin? provider honesty about Lean?) and name the lecture that closes each gap.\n",
|
|
"- The provenance block says the provider checked itself. Why must the agent still run its own inclusion check rather than trust that field?\n"
|
|
]
|
|
}
|
|
],
|
|
"metadata": {
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"name": "python",
|
|
"pygments_lexer": "ipython3"
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 5
|
|
}
|