Crisis BFT consensus protocol — Go PoC, Python recorder, and CrisisViz: a native macOS scrubbable curriculum visualizer (10 chapters, ~18 minutes at 1×, signed-speed slider with reverse playback).
Find a file
saymrwulf a1064660d5 Decentralize crisis_agents: agents own graphs, detect locally, vote by quorum
The previous design routed every Crisis message through a `Mothership` that
also held every agent's LamportGraph, ran the byzantine scan from a
privileged vantage, and built proofs from its own view. That made the
mothership a chokepoint — exactly what a BFT layer is supposed to remove.
This commit redistributes responsibility along the lines you'd expect from
a real open protocol:

Each `CrisisAgent` now owns:
  - its own `LamportGraph` (the agent's view of the network)
  - `emit_claim(claim) → Message`: wraps a Claim into a fully-valid Crisis
    Message built from the agent's OWN graph state, with chain link + cross
    references + mined PoW nonce
  - `receive(message)`: extends my graph if integrity holds; idempotent
  - `gossip_to(peer) → int`: shares everything I have with peer until
    quiescence (Algorithm 4 in the paper, in-process flavor)
  - `detect_mutations() → list[LocalAlarm]`: scans MY graph for same-id
    spacelike vertex pairs via the existing
    `LamportGraph.find_mutations`, filtered by application-layer
    `statement_id` so cross-detector AlarmClaims canonicalize

The `Mothership` shrinks to coordinator-only:
  - bootstrap (register honest agents; trigger boundary open with a joiner)
  - clock (call each agent's `next_turn()` per turn)
  - first-hop routing (sender's emission → declared target subset)
  - all-pairs gossip rounds between turns
  - emit_alarms_from_detectors(): poll each agent for its LocalAlarms,
    wrap any returned alarms into AlarmClaim payloads, broadcast them as
    Crisis Messages over the gossip layer

Gone (regression-tested in `test_no_chokepoint.py`):
  - `Mothership._graphs`, `Mothership.all_graphs()`, `Mothership.graph_of()`
  - `alarm.scan_for_mutations(mothership)`
  - any path where the mothership reads an agent's internal state

New voting layer (`crisis_agents/vote.py`):
  - `AlarmClaim`: a Crisis-payload dataclass discriminated by `kind="alarm"`.
    Wraps the accused process_id, statement_id, witness_digests, and
    detection turn. Round-trips through JSON same as Claim.
  - `quorum_for(n) = ceil(2n/3)`: classic BFT threshold.
  - `tally_alarms(graph, threshold)`: groups AlarmClaim vertices by
    (accused, statement_id, witness_pair), counts unique signer
    process_ids, ratifies groups meeting the threshold. Deterministic
    ordering so two equal graphs produce equal `RatifiedAlarm` lists.
  - `RatifiedAlarm`: the network-level consensus on byzantine behavior.

Multi-signer proofs (`crisis_agents/proof.py`):
  - schema_version bumped 1 → 2.
  - ProofDocument now embeds every signer's process_id_hex and the
    quorum threshold that was met. Self-consistency check enforces
    distinct signers, witness pairs, and signer count ≥ threshold.

Byzantine scenario rewrite:
  - `MockByzantineAgent` now takes an `intro_claim` for its first turn (a
    benign broadcast). The intro is technically necessary: the agent's two
    contradictory variants both chain to the intro vertex, so they can
    propagate through gossip — without it, the second variant would fail
    the chain constraint in any graph already holding the first.
  - `fact_check` scenario: closed phase still has 3 honest agents emitting
    6 claims each into the closed log; Crisis phase grew to 2 turns (intro
    + equivocation) so the byzantine can establish its same-id anchor
    before equivocating.

End-to-end CLI output reframed around six phases:
  1. closed team (no Crisis)
  2. boundary opens
  3. emission + gossip
  4. decentralized detection (each agent reports its own findings)
  5. alarms emitted + gossiped + ratified by quorum
  6. proof emission

Tests (51 fresh + 5 carried over for boundary):
  - `test_mothership.py`: per-agent graph ownership, broadcast vs.
    targeted delivery semantics, gossip propagation, regression guards
    against the removed centralization attributes.
  - `test_alarm.py`: every honest agent independently detects the same
    mutation; the byzantine doesn't detect itself; witness pairs are
    canonical across detectors.
  - `test_vote.py`: AlarmClaim round-trip, quorum formulas, tally
    determinism, mothership convenience method matches direct tallying.
  - `test_proof.py`: build_proof from RatifiedAlarm; multi-signer JSON
    round-trip; tampered-witness/below-quorum/duplicate-signer rejection.
  - `test_no_chokepoint.py` (the centerpiece): after the full lifecycle,
    every honest agent's ratified-alarm set is byte-identical. A single
    byzantine accuser alone cannot ratify. Forbidden attributes don't
    exist on Mothership.

Full suite: 163 tests, all green in 0.80s.

CrisisViz: untouched by this refactor. The `crisis_data.json` pipeline
the visualizer consumes is produced by the orthogonal
`crisis.demo.Simulation`, which this commit doesn't touch.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 21:55:49 +02:00
CrisisViz Add CrisisViz/package-dmg.sh — distributable DMG installer 2026-05-14 15:52:14 +02:00
src Decentralize crisis_agents: agents own graphs, detect locally, vote by quorum 2026-05-14 21:55:49 +02:00
tests Decentralize crisis_agents: agents own graphs, detect locally, vote by quorum 2026-05-14 21:55:49 +02:00
.gitignore Add macOS .app bundle with native Dock icon and activation policy 2026-04-30 20:21:18 +02:00
Crisis.mirco-richter-2019.pdf Initial implementation of the Crisis protocol (Richter, 2019) 2026-04-23 13:20:30 +02:00
crisis_data.json Add JSON export pipeline + event recorder for visualization 2026-04-30 20:06:21 +02:00
INSTALL.md Add INSTALL.md — clone-to-running on a fresh macOS box 2026-05-14 15:51:39 +02:00
LICENSE Add MIT LICENSE; align pyproject.toml accordingly 2026-05-14 15:51:19 +02:00
pyproject.toml Add crisis_agents — Crisis as a coordination layer for AI agent teams 2026-05-14 16:38:11 +02:00
README.md Rewrite parent README — five-layer architecture + audience-shaped quick start 2026-05-14 15:51:31 +02:00

crisis

A proof-of-concept and educational artifact for Mirco Richter's Crisis paper — a DAG-based BFT consensus protocol that achieves total order on messages in fully open, unstructured peer-to-peer networks through virtual voting: votes are never sent explicitly but are deduced from the causal relationships encoded in Lamport graphs.

This repository contains:

  • a Python implementation of the protocol (src/, tests/),
  • an event recorder that exports a deterministic simulation run to JSON,
  • CrisisViz — a native macOS / SwiftUI curriculum visualizer that walks the protocol end-to-end across ten chapters: cast intro, gossip mechanics, partition, round derivation, virtual voting, leader election, total order, the data-availability problem, erasure-coded recovery, and Byzantine fork detection.

Everything in the visualizer is in extreme slow motion and serialized for didactic clarity. A signed speed slider scrubs the chapter forward and backward at any rate from -16\times to +16\times; narration is bound to whichever beat the playhead is on.


Architecture at a glance

flowchart TD
    Paper["📄 <b>Paper — the spec</b><br/>Crisis.mirco-richter-2019.pdf"]
    Paper --> Algos

    subgraph Algos["🧮 Pure protocol algorithms — <code>src/crisis/</code>"]
        direction LR
        Crypto["crypto.py"]
        Msg["message.py"]
        Graph["graph.py"]
        Weight["weight.py"]
        Rounds["rounds.py"]
        Voting["voting.py"]
        Order["order.py"]
    end

    Algos --> RealRT
    Algos --> SimRT

    subgraph RealRT["🌐 <b>Real runtime — <code>node.py</code> + <code>gossip.py</code></b><br/><i>scalable, deployable</i>"]
        Node["CrisisNode<br/>asyncio · TCP push/pull gossip<br/>3 concurrent loops<br/>CLI: <code>crisis-node</code>"]
    end

    subgraph SimRT["🧪 <b>In-process toy runtime — <code>demo.py</code></b><br/><i>deterministic, recordable</i>"]
        SimNode["SimulatedNode<br/>direct in-memory message passing<br/>NetworkParams: delays / drops / silences"]
        SimCtl["Simulation controller<br/>spins up N honest + K byzantine<br/>CLI: <code>crisis-demo</code>"]
        SimNode --- SimCtl
    end

    SimRT --> Rec
    Rec["📼 <b>Recorder — <code>recorder.py</code></b><br/>instruments every algorithm call<br/>captures events + per-step snapshots"]
    Rec --> Export
    Export["📦 <b>JSON exporter — <code>export_json.py</code></b><br/>writes <code>crisis_data.json</code>"]
    Export --> Viz

    subgraph Viz["🎬 <b>CrisisViz — native macOS / SwiftUI</b>"]
        Player["Keynote-style player<br/>10 chapters · ~18 min @ 1×<br/>scrubbable 16× to +16×"]
        Testbed["Testbed harness<br/>invariants · source audit<br/>PNG sweep · 36 MP4 clips"]
    end

    classDef paper fill:#fdf6e3,stroke:#586e75,color:#073642
    classDef pure fill:#eee8d5,stroke:#586e75,color:#073642
    classDef real fill:#fce5cd,stroke:#cc4125,color:#660000
    classDef sim fill:#d9ead3,stroke:#38761d,color:#0b3d0b
    classDef rec fill:#cfe2f3,stroke:#2c5f8f,color:#062b4d
    classDef viz fill:#ead1dc,stroke:#741b47,color:#3d0a26
    class Paper paper
    class Algos pure
    class RealRT real
    class SimRT sim
    class Rec,Export rec
    class Viz viz

Key architectural fact — the recording pipeline that feeds CrisisViz only exercises the SimulatedNode path (in-process, deterministic, in-memory message passing). The CrisisNode TCP runtime is a separately developed PoC of how a real network deployment would look; it is not what produces crisis_data.json. The two runtimes are siblings, not layers.


Repository layout

crisis/                                       ← git root
├── Crisis.mirco-richter-2019.pdf             the paper
├── README.md                                  this file
├── INSTALL.md                                 fresh-macOS install guide
├── LICENSE                                    MIT (code only; paper is CC-BY-4.0)
├── pyproject.toml                             Python ≥3.11, networkx, pytest
├── crisis_data.json                           simulation export (source of truth)
│
├── src/crisis/                                ── PROTOCOL PoC (Python) ──
│   ├── crypto.py, message.py                  random-oracle hash + Message/Vertex
│   ├── graph.py, weight.py, rounds.py         Lamport DAG + PoW weight + round derivation
│   ├── voting.py, order.py                    BBA virtual voting + total order
│   ├── gossip.py, node.py                     real TCP runtime (CrisisNode)
│   ├── demo.py                                in-process simulation harness
│   ├── recorder.py                            event instrumentation
│   └── export_json.py                         JSON exporter for CrisisViz
├── tests/                                     pytest suite
│
└── CrisisViz/                                 ── VISUALIZER (Swift / macOS 26) ──
    ├── Package.swift, bundle.sh, package-dmg.sh
    ├── Sources/CrisisViz/                     App, Engine, Model, Chapters, Views, Glass, Testbed, Canvas
    ├── README.md                              Swift-side human guide
    └── HANDOFF.md                             agent-to-agent engineering log

Quick start

There are three audiences. Pick the one that matches what you want to do.

🧮 Verify the protocol — pytest

cd crisis
source .venv/bin/activate    # set up per INSTALL.md if first time
pytest -q

Runs the algorithm unit tests (crypto, graph, rounds, weight, message, order, voting, recorder, simulation). Should be green in under a second.

🧪 Run a deterministic simulation — Python CLI

python -m crisis.demo --nodes 4 --byzantine 1 --rounds 10

Spins up four honest + one byzantine SimulatedNode, runs ten consensus rounds in-process with a deterministic seed, prints the resulting total order. To export a fresh crisis_data.json for CrisisViz:

python -m crisis.export_json --steps 80 -o crisis_data.json
cp crisis_data.json CrisisViz/Sources/CrisisViz/crisis_data.json

🎬 Watch the visualizer — Swift / macOS

cd CrisisViz
./bundle.sh          # builds CrisisViz.app and opens it
# or:
./package-dmg.sh     # builds CrisisViz.dmg for distribution

Then arrow keys ←/→ to navigate, Space to play/pause, the bottom slider to scrub at any signed speed from -16\times to +16\times.


  • INSTALL.md — clone-to-running on a fresh macOS box. Prerequisites, Python venv setup, Swift toolchain, regenerating sim data, troubleshooting.
  • CrisisViz/README.md — Swift-side guide: serial-timeline pattern, testbed outputs, controls, cast convention.
  • CrisisViz/HANDOFF.md — engineering log for the next coding agent: current state, architecture pointers, hard-won rules.

License

  • Code (src/, tests/, CrisisViz/) is licensed under the MIT License.
  • Paper (Crisis.mirco-richter-2019.pdf) by Mirco Richter is a separately licensed artifact under CC-BY-4.0.