proof-aware-crypto-tooling-agent/notebooks/06_merkle_transparency_logs.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Lecture 6: Merkle Transparency Logs\n",
        "\n",
        "Transparency logs make signed statements auditable. PACTA uses an RFC 9162-style Merkle accumulator over signed proof-check attestations. A provider signs the tree head, and an agent verifies an inclusion proof before acting on the attestation.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Learning Objectives\n",
        "\n",
        "- Implement leaf and node hashing with domain separation.\n",
        "- Compute a Merkle root.\n",
        "- Generate and verify inclusion proofs.\n",
        "- Generate and verify consistency proofs.\n",
        "- Explain Signed Tree Heads and signature policy.\n",
        "- Explain why ML-DSA must fail closed when unavailable.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## RFC 9162 Hash Shape\n",
        "\n",
        "PACTA follows the Certificate Transparency hash structure:\n",
        "\n",
        "- Empty tree hash: `SHA256(\"\")`\n",
        "- Leaf hash: `SHA256(0x00 || leaf_input)`\n",
        "- Node hash: `SHA256(0x01 || left || right)`\n",
        "\n",
        "The prefix bytes prevent a leaf value from being confused with an internal node value.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from pathlib import Path\n",
        "import sys\n",
        "\n",
        "repo_root = Path.cwd()\n",
        "if not (repo_root / \"src\" / \"pacta\").exists():\n",
        "    repo_root = repo_root.parent\n",
        "sys.path.insert(0, str(repo_root / \"src\"))\n",
        "\n",
        "from pacta.transparency import (\n",
        "    leaf_hash,\n",
        "    node_hash,\n",
        "    merkle_root,\n",
        "    inclusion_proof,\n",
        "    verify_inclusion,\n",
        "    consistency_proof,\n",
        "    verify_consistency,\n",
        ")\n",
        "\n",
        "leaves = [f\"attestation-{i}\".encode() for i in range(1, 6)]\n",
        "root = merkle_root(leaves)\n",
        "print(root.hex())\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "for index, leaf in enumerate(leaves):\n",
        "    proof = inclusion_proof(leaves, index)\n",
        "    ok = verify_inclusion(leaf, index, len(leaves), proof, root)\n",
        "    print(index, ok, [node.hex()[:12] for node in proof])\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Consistency Proofs\n",
        "\n",
        "An inclusion proof answers: \"Is this leaf in this tree?\"\n",
        "\n",
        "A consistency proof answers: \"Is the newer tree an append-only extension of the older tree?\"\n",
        "\n",
        "Both are needed for a monitored transparency system. Inclusion is enough for one agent to bind one attestation to one signed tree head. Consistency lets monitors detect equivocation or tree rewrites across time.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "old_size = 3\n",
        "old_root = merkle_root(leaves[:old_size])\n",
        "new_root = merkle_root(leaves)\n",
        "proof = consistency_proof(leaves, old_size)\n",
        "print(\"old:\", old_root.hex())\n",
        "print(\"new:\", new_root.hex())\n",
        "print(\"proof:\", [node.hex()[:12] for node in proof])\n",
        "print(\"consistent:\", verify_consistency(old_size, len(leaves), old_root, new_root, proof))\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Signed Tree Heads\n",
        "\n",
        "A Signed Tree Head records:\n",
        "\n",
        "- log ID,\n",
        "- tree size,\n",
        "- timestamp,\n",
        "- root hash,\n",
        "- hash algorithm,\n",
        "- signatures.\n",
        "\n",
        "PACTA signs the canonical JSON STH payload with Ed25519 through OpenSSL. It also records an ML-DSA-65 slot. On this host, if no real ML-DSA backend is present, the slot is `unavailable`.\n",
        "\n",
        "Policy matters:\n",
        "\n",
        "- `require-signatures ed25519`: verify Ed25519 and allow ML-DSA to be unavailable.\n",
        "- `require-signatures both`: require Ed25519 and ML-DSA verified. If ML-DSA is unavailable, fail closed.\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
        "from pacta.postquantum import detect_ml_dsa\n",
        "\n",
        "capability = detect_ml_dsa()\n",
        "print(capability.available)\n",
        "print(capability.backend)\n",
        "print(capability.reason)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Why Ed25519 and ML-DSA Together?\n",
        "\n",
        "Ed25519 is useful because it is widely deployed, fast, and directly relevant to the Ed25519 proof corpus. That creates a deliberate \"eat your own dogfood\" loop: the proof-checking ecosystem signs evidence using a primitive whose implementation family is under formal scrutiny.\n",
        "\n",
        "ML-DSA adds post-quantum robustness for the accumulator signature layer. But it must be a real signature, not an aspirational label. If a host lacks ML-DSA, the correct result is an explicit blocker.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Exercises\n",
        "\n",
        "- Tamper with one leaf and show that inclusion verification fails.\n",
        "- Explain why the tree head signature must cover tree size as well as root hash.\n",
        "- Write a policy for when an autonomous agent should require `both` signatures.\n",
        "- Research checkpoint: compare PACTA's local prototype to production Certificate Transparency monitor/gossip requirements.\n"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "name": "python",
      "pygments_lexer": "ipython3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}