Update all documentation for Go daemon rewrite

All docs, README, and presentation now reflect the Go daemon architecture: Python/FastAPI/LanceDB/PyMuPDF references replaced with Go/chi/SQLite/pdftotext. Updated test counts (97), model names (qwen3-4b-2507), app bundle structure, installer steps, and tech stack tables.
2026-05-14 20:47:51 +00:00 · 2026-02-13 19:29:23 +01:00 · 2026-02-13 19:29:23 +01:00 · 9dfb9ff684
commit 9dfb9ff684
parent c34f1edab2
6 changed files with 165 additions and 205 deletions
--- a/README.md
+++ b/README.md
@ -1,42 +1,37 @@
 # Knowledge Refinery

-A local-first macOS Tahoe application that ingests heterogeneous document corpora, extracts structured knowledge via local LLMs (LM Studio), and provides semantic search with 3D concept visualization.
+A local-first macOS application that ingests heterogeneous document corpora, extracts structured knowledge via local LLMs (LM Studio), and provides semantic search with 3D concept visualization.

 ## Installation

 ### Prerequisites
- **macOS Tahoe** (26.x) on Apple Silicon
+- **macOS 15+** (Sequoia or later) on Apple Silicon
 - **Xcode** or Xcode Command Line Tools (for Swift 6.2+)
- **Python 3.12+** (system Python or from python.org)
+- **Go 1.22+** from [go.dev](https://go.dev/dl/) or `brew install go`
 - **LM Studio** from [lmstudio.ai](https://lmstudio.ai)

 ### One-Line Install

 ```bash
-git clone <repo-url> && cd LongLocalTimeHorizonInfoRetrieval && bash scripts/install.sh
+git clone https://github.com/saymrwulf/KnowledgeRefinery.git && cd KnowledgeRefinery && bash scripts/install.sh
 ```

 This will:
-1. Check all prerequisites
-2. Create a Python virtual environment and install dependencies
-3. Build the SwiftUI application
-4. Create a proper `.app` bundle
-5. Install to `/Applications`
+1. Check all prerequisites (Go, Swift, Xcode)
+2. Run all Go daemon tests (89 tests)
+3. Build the Go daemon + SwiftUI app into a `.app` bundle
+4. Install to `/Applications`
+5. Create the data directory at `~/.knowledge-refinery/`

 ### Manual Build

 ```bash
-# Set up daemon
-cd daemon
-python3 -m venv .venv
-.venv/bin/pip install -e ".[dev]"
-
-# Build app bundle
-cd ..
+# Build Go daemon + .app bundle
 make build

-# Or just run in development mode
-make app-run
+# Or run in development mode
+make daemon       # Build Go daemon binary
+make app-run      # Run SwiftUI app (builds daemon first)
 ```

 ### LM Studio Setup
@ -44,7 +39,7 @@ make app-run
 Before launching Knowledge Refinery:
 1. Open LM Studio
 2. Load models:
-   - **Chat**: `gemma-3-4b` (or any chat model)
+   - **Chat**: `qwen3-4b-2507` (or any small chat model)
   - **Embeddings**: `nomic-embed-text-v1.5` (768-dim)
 3. Start the local server on port **1234**

@ -52,26 +47,25 @@ Before launching Knowledge Refinery:

 1. Launch **Knowledge Refinery** from Applications or Spotlight
 2. The dashboard shows LM Studio status (green = connected)
-3. Click **New Workspace** — name it, add data lake folders
-4. Click **Start All** to launch all workspace daemons and auto-start ingestion
-5. Watch live pipeline progress: stage tracker, animated counters, activity log
-6. Search, explore the concept universe, browse clusters
+3. Click **New Workspace** — name it, add source folders
+4. Open the workspace and click **Process Documents** to run the pipeline
+5. Watch live progress: stage tracker, animated counters, activity log
+6. Search, explore the concept universe, browse themes

 ## Architecture

- **SwiftUI Master Control App** — Multi-workspace dashboard, LM Studio monitoring, daemon lifecycle, live pipeline visibility
- **Python Daemon** (FastAPI) — Per-workspace instances with independent ports and data directories (`~/.knowledge-refinery/workspaces/<id>/`)
- **Live Pipeline Progress** — 1.5s fast polling during ingestion, enriched `/ingest/status` with per-stage progress, counters, and activity log
- **LanceDB** — Embedded vector store for semantic search
- **SQLite** — Metadata, graph store, pipeline state
- **LM Studio** — Local LLM inference (embeddings + chat)
- **WebGPU** — 3D concept universe visualization with auto-refresh during ingestion
+- **SwiftUI App** — Multi-workspace dashboard, LM Studio monitoring, daemon lifecycle, live pipeline visibility
+- **Go Daemon** (chi router) — 11MB single binary, zero dependencies, per-workspace instances on independent ports
+- **SQLite** — All storage: metadata, vectors (as BLOBs with brute-force cosine search), graph, pipeline state
+- **6-Stage Pipeline** — scan, extract, chunk, embed, annotate, conceptualize
+- **LM Studio** — Local LLM inference at `127.0.0.1:1234` (embeddings + chat)
+- **WebGPU / Canvas2D** — 3D concept universe visualization with interactive fallback

 ## Project Structure

 ```
 apps/macos/KnowledgeRefinery/   SwiftUI macOS application
-daemon/                         Python backend daemon
+daemon-go/                      Go daemon (chi, SQLite, tiktoken)
 shared/                         Prompt templates, schemas
 docs/                           Architecture and operational docs
 scripts/                        Build and install scripts
@ -83,19 +77,16 @@ dist/                           Built .app bundle (after make build)

 ```bash
 make help          # Show all commands
-make test          # Run daemon tests + Swift build check
+make test          # Run 89 Go tests + Swift build check
 make app-run       # Run app via swift run (dev mode)
-make daemon-run    # Run daemon directly
+make daemon-run    # Run daemon standalone
 make clean         # Remove build artifacts
 ```

 ## Milestones

- **M1**: Core ingestion + search + evidence
- **M2**: LLM structured annotation
- **M3**: Concept clustering + labeling
- **M4**: WebGPU 3D Universe visualization
- **M5**: Semantic zoom + lenses
- **M6**: Extended format support (EPUB, archives, DICOM)
+- **M1-M6**: Core pipeline, search, annotation, clustering, WebGPU, extended formats
 - **M7**: Master Control App (multi-workspace, LM Studio monitoring, daemon lifecycle)
- **M8**: Live Pipeline Visibility (real-time progress panel, activity log, universe auto-refresh)
+- **M8**: Live Pipeline Visibility (real-time progress, activity log, universe auto-refresh)
+- **M9-M10**: UX language overhaul, macOS sizing improvements
+- **M11**: Go daemon rewrite (Python replaced with single Go binary, 89 tests)
--- a/docs/architecture.md
+++ b/docs/architecture.md
@ -13,29 +13,29 @@ Knowledge Refinery is a local-first macOS application that ingests heterogeneous
 │  │ Search │ Evidence    │         │
 │  │ View   │ Panel (QL)  │         │
 │  ├────────┼────────────┤         │
-│  │Pipeline│ Volume      │         │
-│  │Progress│ Manager     │         │
+│  │Pipeline│ Source      │         │
+│  │Progress│ Folders     │         │
 │  │ Panel  │             │         │
 │  └────────┴────────────┘         │
-│       │ HTTP (localhost)            │
-│       │ ┌──────────────────┐      │
-│       │ │ 1.5s poll loop   │      │
-│       │ │ /ingest/status   │◀─┐   │
-│       │ └──────────────────┘  │   │
-│       │    auto-stop on done  │   │
-│       │                       │   │
-│       │ ┌──────────────────┐  │   │
-│       │ │ 5s universe      │  │   │
-│       │ │ auto-refresh     │──┘   │
-│       │ └──────────────────┘      │
+│       │ HTTP (localhost)          │
+│       │ ┌──────────────────┐     │
+│       │ │ 1.5s poll loop   │     │
+│       │ │ /ingest/status   │◀─┐  │
+│       │ └──────────────────┘  │  │
+│       │    auto-stop on done  │  │
+│       │                       │  │
+│       │ ┌──────────────────┐  │  │
+│       │ │ 5s universe      │  │  │
+│       │ │ auto-refresh     │──┘  │
+│       │ └──────────────────┘     │
 └───────┼──────────────────────────┘
        ▼
 ┌──────────────────────────────────┐
-│   Refinery Daemon (Python)       │
+│   Go Daemon (11MB binary)        │
 │   Per-workspace on independent   │
 │   port + data dir                │
 │  ┌──────────────────────┐        │
-│  │  FastAPI Server      │        │
+│  │  chi Router + CORS   │        │
 │  └──────────┬───────────┘        │
 │             ▼                    │
 │  ┌──────────────────────┐        │
@ -54,10 +54,11 @@ Knowledge Refinery is a local-first macOS application that ingests heterogeneous
 │  │    (200-entry buf)   │        │
 │  └──────────────────────┘        │
 │                                  │
-│  ┌─────────┬───────────┐         │
-│  │ SQLite  │ LanceDB   │         │
-│  │ (meta)  │ (vectors) │         │
-│  └─────────┴───────────┘         │
+│  ┌──────────────────────┐        │
+│  │ SQLite (WAL mode)    │        │
+│  │  metadata + vectors  │        │
+│  │  + graph + state     │        │
+│  └──────────────────────┘        │
 └───────┼──────────────────────────┘
        ▼
 ┌──────────────────────────────────┐
@ -78,16 +79,16 @@ Knowledge Refinery is a local-first macOS application that ingests heterogeneous

 ## Data Flow

-Files → FileAsset → ContentAtom → Chunk → Embedding (LanceDB) + Annotation (SQLite)
-                                                        ↓
-                                              ConceptNode + GraphEdge
+Files → FileAsset → ContentAtom → Chunk → Vector (SQLite BLOB) + Annotation
+                                                      ↓
+                                            ConceptNode + GraphEdge

-## Live Progress Data Flow (M8)
+## Live Progress Data Flow

 During pipeline execution, the daemon maintains in-memory state that the app polls:

 ```
-Pipeline Orchestrator
+Pipeline Orchestrator (goroutine)
    │
    ├──▶ live progress dict (per-stage status: pending/running/done)
    │       stage_name, progress_pct, item_count
@ -101,7 +102,7 @@ SwiftUI App polling loop (1.5s interval):
    GET /ingest/status ──▶ stages, counters, activity_log
    │
    ├── Pipeline Progress Panel: checkmarks + progress bars per stage
-    ├── Animated counters: chunks, vectors, annotations, concepts, edges
+    ├── Animated counters: passages, indexed, insights, themes, links
    ├── Interaction indicators: App↔Daemon, Daemon↔LM Studio
    ├── Auto-scrolling activity log
    └── Auto-stop polling when pipeline status = idle/done
@ -112,10 +113,13 @@ Universe auto-refresh (5s timer during ingestion):

 ## Key Design Decisions

- **LanceDB** over Qdrant: Embedded, no separate server, local-first
- **SQLite** for metadata/graph: Simple, reliable, WAL mode for concurrency
+- **Go single binary** over Python: Zero dependencies, instant startup, 11MB, no venv/pip issues
+- **SQLite for everything**: Metadata, vectors (as BLOBs with brute-force cosine search), graph — one file, WAL mode
+- **chi router**: Lightweight HTTP routing with path params, CORS middleware
+- **modernc.org/sqlite**: Pure Go SQLite driver, no CGo, true single binary
+- **tiktoken-go**: Accurate token counting matching OpenAI tokenizer
 - **Deterministic chunk IDs**: SHA-256(asset_id + anchor + normalized_text_hash)
 - **Versioned annotations**: Never overwrite, mark active by pipeline version
 - **Evidence-native**: Every derived insight links back to source file + location
- **Fast polling over WebSocket**: 1.5s HTTP polls are simpler and sufficient for pipeline status; avoids connection lifecycle complexity
- **Ring buffer for activity log**: Fixed 200-entry buffer prevents memory growth during long pipeline runs
+- **Fast polling over WebSocket**: 1.5s HTTP polls are simpler and sufficient for pipeline status
+- **Ring buffer for activity log**: Fixed 200-entry buffer prevents memory growth during long runs
--- a/docs/data-model.md
+++ b/docs/data-model.md
@ -14,7 +14,8 @@ ContentAtom                 │
    ▼ (split into)          │
 Chunk ──────────────────────┤
    │                       │
-    ├──▶ Vector (LanceDB)   │
+    ├──▶ Vector (SQLite)    │
+    │    768-dim BLOB       │
    │                       │
    ├──▶ Annotation         │
    │    (versioned)        │
@ -26,7 +27,7 @@ Chunk ──────────────────────┤
    (hierarchical)
 ```

-## Tables
+## Tables (SQLite)

 ### file_assets
 Tracks every file in watched volumes. Status progresses through:
@ -38,7 +39,21 @@ Each atom has an evidence_anchor linking to exact source location.

 ### chunks
 Deterministic text segments (500-800 tokens). IDs are stable across re-processing.
-Linked to LanceDB vectors via embedding_id.
+Linked to vectors in `chunk_vectors` table via chunk ID.
+
+### chunk_vectors
+Embedding vectors stored as binary BLOBs (768 x float32 = 3072 bytes per vector).
+Loaded into memory at startup for brute-force cosine similarity search.
+
+| Field | Type | Description |
+|-------|------|-------------|
+| id | TEXT PRIMARY KEY | Matches chunks.id |
+| vector | BLOB | 768-dim float32 embedding |
+| text | TEXT | Chunk text |
+| asset_id | TEXT | Source file |
+| asset_path | TEXT | File path |
+| evidence_anchor | TEXT | JSON anchor |
+| pipeline_version | TEXT | Version tag |

 ### annotations
 LLM-generated structured metadata per chunk. **Never overwritten** - new annotations
@ -56,7 +71,7 @@ Each edge stores evidence references back to source chunks.
 ### pipeline_jobs
 Crash recovery: tracks job state so processing resumes after restart.

-## Live Progress State (M8, In-Memory)
+## Live Progress State (In-Memory)

 During pipeline execution, the daemon maintains ephemeral in-memory structures that are not persisted to SQLite:

@ -74,19 +89,10 @@ Per-stage status object returned in the `live` field of `/ingest/status`:
 }
 ```

-Each stage transitions through `pending` -> `running` -> `done`.
-
 ### Activity Log Ring Buffer
-A fixed-size circular buffer (200 entries) that records pipeline events. The API returns the most recent 50 entries. Each entry contains a timestamp and message string:
-
-```json
-{"timestamp": "2026-02-12T10:30:03Z", "message": "Found 47 files, 12 new"}
-```
-
-The ring buffer prevents unbounded memory growth during long pipeline runs. It is reset at the start of each new pipeline execution.
+A fixed-size circular buffer (200 entries) that records pipeline events. The API returns the most recent 50 entries.

 ### Enriched Status Counters
-The `/ingest/status` response includes running totals updated as each stage completes:

 | Counter | Description |
 |---------|-------------|
@ -111,17 +117,3 @@ Every derived artifact links back to source via JSON evidence anchors:
    "line_end": 58
 }
 ```
-
-## Vector Schema (LanceDB)
-
-| Field | Type | Description |
-|-------|------|-------------|
-| id | string | Matches chunks.id |
-| vector | float32[] | Embedding vector |
-| text | string | Chunk text |
-| asset_id | string | Source file |
-| asset_path | string | File path |
-| evidence_anchor | string | JSON anchor |
-| topics | string | Comma-separated topics |
-| atom_type | string | text/image/etc |
-| pipeline_version | string | Version tag |
--- a/docs/operational-notes.md
+++ b/docs/operational-notes.md
@ -7,11 +7,9 @@ Each workspace has its own data directory under `~/.knowledge-refinery/workspace
 | Item | Path |
 |------|------|
 | Workspace root | `~/.knowledge-refinery/workspaces/<id>/` |
-| SQLite DB | `~/.knowledge-refinery/workspaces/<id>/refinery.db` |
-| Vector DB | `~/.knowledge-refinery/workspaces/<id>/vectors/` |
-| Thumbnails | `~/.knowledge-refinery/workspaces/<id>/thumbnails/` |
-| Temp files | `~/.knowledge-refinery/workspaces/<id>/tmp/` |
+| SQLite DB (metadata + vectors + graph) | `~/.knowledge-refinery/workspaces/<id>/refinery.db` |
 | PID file | `~/.knowledge-refinery/workspaces/<id>/daemon.pid` |
+| Workspace config | `~/.knowledge-refinery/workspaces.json` |

 ## Resetting

@ -22,12 +20,12 @@ rm -rf ~/.knowledge-refinery

 ## Monitoring

-The daemon logs to stdout. Key log patterns:
- `Stage N: ...` - Pipeline stage progress
- `Embedded batch N: X chunks` - Embedding progress
- `ERROR` - Errors during processing
+The Go daemon logs to stdout with structured messages. Key log patterns:
+- `[pipeline] Stage: ...` - Pipeline stage progress
+- `[embedder] Embedded batch: X chunks` - Embedding progress
+- `[error]` - Errors during processing

-### Live Pipeline Monitoring (M8)
+### Live Pipeline Monitoring

 During pipeline execution, real-time progress is available via the enriched `/ingest/status` endpoint. The daemon maintains:

@ -35,7 +33,7 @@ During pipeline execution, real-time progress is available via the enriched `/in
 - **Counters**: chunk_count, annotation_count, concept_count, edge_count
 - **Activity log**: 200-entry ring buffer; the last 50 events are returned via the API

-The SwiftUI app polls at 1.5-second intervals and renders a full Pipeline Progress Panel with stage checkmarks, animated counters, and an auto-scrolling activity log. Polling auto-stops when the pipeline reaches idle/done state. The 3D universe auto-refreshes every 5 seconds during ingestion using `mergeUniverse()` for incremental node injection.
+The SwiftUI app polls at 1.5-second intervals and renders a Pipeline Progress Panel with stage checkmarks, animated counters, and an auto-scrolling activity log. Polling auto-stops when the pipeline reaches idle/done state. The universe visualization auto-refreshes every 5 seconds during processing using `mergeUniverse()` for incremental node injection.

 ## API Endpoints

@ -48,19 +46,21 @@ The SwiftUI app polls at 1.5-second intervals and renders a full Pipeline Progre
 | POST | /ingest/start | Start pipeline |
 | GET | /ingest/status | Pipeline status |
 | POST | /search | Vector search |
-| GET | /search/quick?q=... | Quick search |
 | GET | /evidence/{asset_id} | Get asset info |
 | GET | /evidence/chunk/{chunk_id} | Get chunk details |
 | GET | /evidence/assets/all | List all assets |
-| GET | /universe/snapshot | Universe snapshot |
+| GET | /universe/snapshot?lod=macro | Universe snapshot |
 | POST | /universe/focus | Focus on node |
 | POST | /concepts/refine | Refine concept |
 | GET | /concepts/list | List concepts |
+| GET | /concepts/{id} | Concept detail |

 ## Performance Considerations

 - Large files (>500MB) are skipped by default
- Embedding batch size defaults to 32 (adjustable)
+- Embedding batch size defaults to 32
 - SQLite uses WAL mode for concurrent reads
- Pipeline runs in a background thread
+- Pipeline runs in a background goroutine
 - Incremental processing skips unchanged files (content hash comparison)
+- Vector search: brute-force cosine similarity, all vectors loaded in memory (~150MB for 50K vectors)
+- Go daemon starts in <100ms, uses ~30MB base memory
--- a/docs/running.md
+++ b/docs/running.md
@ -2,36 +2,40 @@

 ## Prerequisites

- macOS Tahoe (26.x)
- Python 3.12+
- Xcode 26+
- LM Studio running locally with at least one model loaded
+- macOS 15+ (Sequoia or later)
+- Go 1.22+
+- Xcode or Xcode Command Line Tools (Swift 6.2+)
+- LM Studio running locally with models loaded

 ## 1. Start LM Studio

 1. Open LM Studio
-2. Load an embedding model (e.g., `nomic-embed-text-v1.5` or `text-embedding-3-small`)
-3. Load a chat model (e.g., `llama-3.2-3b-instruct` or similar)
+2. Load an embedding model: `nomic-embed-text-v1.5` (768-dim)
+3. Load a chat model: `qwen3-4b-2507` (or similar small model)
 4. Start the local server (default port 1234)
 5. Verify: `curl http://127.0.0.1:1234/v1/models`

 ## 2. Start the Daemon

+### Via the App (Recommended)
+
+The SwiftUI app auto-starts daemons for all workspaces on launch. Each workspace runs an independent Go daemon with its own port and data directory.
+
+### Manually
+
 ```bash
-cd daemon
-source .venv/bin/activate
-python -m knowledge_refinery.main
+cd daemon-go
+go build -o knowledge-refinery-daemon .
+./knowledge-refinery-daemon
 ```

 The daemon will:
 - Create data directory at `~/.knowledge-refinery/workspaces/<id>/`
- Initialize SQLite database
+- Initialize SQLite database (metadata + vectors)
 - Connect to LM Studio
 - Write a PID file to `{data_dir}/daemon.pid` for process detection
 - Listen on its assigned port (default `http://127.0.0.1:8742`)

-> **Tip**: Use **Start All** in the app toolbar to launch all workspace daemons at once. Each workspace runs an independent daemon with its own port and data directory. After connection, ingestion auto-starts.
-
 ### Environment Variables

 | Variable | Default | Description |
@ -48,14 +52,15 @@ curl http://127.0.0.1:8742/health

 ## 3. Run the macOS App

+### From .app Bundle
 ```bash
-cd apps/macos/KnowledgeRefinery
-swift run
+make build
+open "dist/Knowledge Refinery.app"
 ```

-Or open in Xcode:
+### Development Mode
 ```bash
-open Package.swift
+make app-run
 ```

 The app will:
@ -64,64 +69,35 @@ The app will:
 - Auto-restart crashed daemons (up to 3 times)
 - Show connection status in the toolbar

-## 4. Ingest Documents
+## 4. Process Documents

-1. In the app, go to **Volumes** tab
+1. In the app, go to **Source Folders** tab
 2. Click **Add Folder** and select a directory
-3. Go to **Ingest** tab and click **Start Ingestion**, or use **Start All** from the dashboard
-4. Watch live pipeline progress in the **Pipeline Progress Panel**:
-   - **Stage tracker**: Each of the 6 stages (Scan, Extract, Chunk, Embed, Annotate, Conceptualize) shows a checkmark when complete or an animated progress bar when running
-   - **Animated counters**: Live tallies for chunks, vectors, annotations, concepts, and edges
+3. Click **Process Documents** to run the pipeline
+4. Watch live pipeline progress in the **Processing Steps** panel:
+   - **Stage tracker**: Each of the 6 stages shows a checkmark when complete or a progress bar when running
+   - **Animated counters**: Live tallies for passages, indexed, insights, themes, and links
   - **Interaction indicators**: Visual status of App-to-Daemon and Daemon-to-LM Studio connections
   - **Activity log**: Auto-scrolling log of the last 50 pipeline events
-5. The dashboard card shows a compact spinner with the current stage name and chunk count
-6. The 3D universe auto-refreshes every 5 seconds during ingestion, using incremental node injection
-
-The app polls `/ingest/status` every 1.5 seconds during pipeline execution and automatically stops polling when the pipeline completes.
+5. The universe auto-refreshes every 5 seconds during processing

 ### Via API

 ```bash
-# Add a volume
+# Add a source folder
 curl -X POST http://127.0.0.1:8742/volumes/add \
  -H "Content-Type: application/json" \
  -d '{"path": "/path/to/documents"}'

-# Start ingestion
+# Start processing
 curl -X POST http://127.0.0.1:8742/ingest/start \
  -H "Content-Type: application/json" \
  -d '{}'

-# Check status (enriched response with live progress)
+# Check status
 curl http://127.0.0.1:8742/ingest/status
 ```

-The enriched `/ingest/status` response includes:
-
-```json
-{
-  "status": "running",
-  "stage": "embed",
-  "chunk_count": 142,
-  "annotation_count": 87,
-  "concept_count": 12,
-  "edge_count": 45,
-  "live": {
-    "scan": {"status": "done", "progress_pct": 100},
-    "extract": {"status": "done", "progress_pct": 100},
-    "chunk": {"status": "done", "progress_pct": 100},
-    "embed": {"status": "running", "progress_pct": 64},
-    "annotate": {"status": "pending", "progress_pct": 0},
-    "conceptualize": {"status": "pending", "progress_pct": 0}
-  },
-  "activity_log": [
-    {"timestamp": "2026-02-12T10:30:01Z", "message": "Scanning 3 volumes..."},
-    {"timestamp": "2026-02-12T10:30:03Z", "message": "Found 47 files, 12 new"},
-    "..."
-  ]
-}
-```
-
 ## 5. Search

 Use the **Search** tab in the app, or:
@ -134,7 +110,8 @@ curl -X POST http://127.0.0.1:8742/search \

 ## Troubleshooting

- **Daemon won't start**: Check that port 8742 is free
+- **Daemon won't start**: Check that port 8742 is free (`lsof -i :8742`)
 - **LM Studio unavailable**: Ensure LM Studio server is running on port 1234
 - **No embeddings**: Verify an embedding model is loaded in LM Studio
 - **App can't connect**: Check daemon is running on the expected port
+- **Build fails**: Ensure Go 1.22+ and Swift 6.2+ are installed
--- a/presentation.html
+++ b/presentation.html
@ -687,7 +687,7 @@ h3 {
      <div class="stat"><div class="stat-num">0</div><div class="stat-label">Cloud Calls</div></div>
    </div>
    <div style="margin-top:2rem; color:var(--fg-subtle); font-size:0.85rem;">
-      macOS Tahoe &middot; SwiftUI &middot; Python 3.14 &middot; FastAPI &middot; LM Studio &middot; LanceDB &middot; WebGPU &middot; Multi-Workspace
+      macOS &middot; SwiftUI &middot; Go &middot; chi &middot; LM Studio &middot; SQLite &middot; WebGPU &middot; Multi-Workspace
    </div>
  </div>
 </div>
@ -787,13 +787,13 @@ h3 {
        <span style="font-size:0.65rem; color:var(--fg-subtle);">127.0.0.1:8742</span>
      </div>
      <div class="arch-box" style="border-color:#7c3aed;">
-        <h4 style="color:#7c3aed;">&#x2699;&#xFE0F; Python Daemon</h4>
+        <h4 style="color:#7c3aed;">&#x2699;&#xFE0F; Go Daemon</h4>
        <div style="font-size:0.85rem; color:var(--fg-muted);">
-          <strong>FastAPI</strong> server with CORS<br>
+          <strong>chi</strong> router with CORS<br>
          <strong>6-Stage Pipeline</strong> orchestrator<br>
-          <strong>SQLite</strong> metadata + graph (WAL mode)<br>
-          <strong>LanceDB</strong> embedded vector store<br>
-          <strong>7 Extractors</strong> + sandbox worker
+          <strong>SQLite</strong> metadata + vectors + graph (WAL)<br>
+          <strong>11MB</strong> single binary, zero deps<br>
+          <strong>7 Extractors</strong> + fallback chain
        </div>
      </div>
    </div>
@ -805,7 +805,7 @@ h3 {
      <div class="arch-box" style="border-color:#059669;">
        <h4 style="color:#059669;">&#x1F916; LM Studio</h4>
        <div style="font-size:0.85rem; color:var(--fg-muted);">
-          <strong>gemma-3-4b</strong> &mdash; Chat / Annotation<br>
+          <strong>qwen3-4b-2507</strong> &mdash; Chat / Annotation<br>
          <strong>nomic-embed-text-v1.5</strong> &mdash; Embeddings<br>
          Running on 127.0.0.1:1234
        </div>
@ -844,7 +844,7 @@ h3 {
      <div class="pipeline-stage">
        <div class="num" style="background:#0891b2;">4</div>
        <span class="label">Embed</span>
-        <span class="desc">768-dim vectors via nomic-embed-text stored in LanceDB</span>
+        <span class="desc">768-dim vectors via nomic-embed-text stored in SQLite</span>
      </div>
      <div class="pipeline-arrow">&rarr;</div>
      <div class="pipeline-stage">
@ -877,7 +877,7 @@ h3 {
      <div class="extractor-item">
        <div class="ext-icon">&#x1F4D1;</div>
        <div class="ext-name">PDF</div>
-        <div class="ext-types">.pdf &middot; Per-page text &middot; PyMuPDF</div>
+        <div class="ext-types">.pdf &middot; Per-page text &middot; pdftotext</div>
        <div><span class="tag tag-accent" style="font-size:0.6rem;">Priority 20</span></div>
      </div>
      <div class="extractor-item">
@ -895,7 +895,7 @@ h3 {
      <div class="extractor-item">
        <div class="ext-icon">&#x1FA7B;</div>
        <div class="ext-name">DICOM</div>
-        <div class="ext-types">.dcm &middot; Medical metadata &middot; Optional pydicom</div>
+        <div class="ext-types">.dcm &middot; Medical metadata &middot; Binary header parsing</div>
        <div><span class="tag tag-accent" style="font-size:0.6rem;">Priority 15</span></div>
      </div>
      <div class="extractor-item">
@ -1076,7 +1076,7 @@ ContentAtom <span class="comment">-- text/image/table/metadata</span>
  <span class="op">&darr;</span> <span class="comment">(split into)</span>
 Chunk <span class="comment">-- deterministic ID, 500-800 tokens</span>
  <span class="op">&darr;</span>
-  <span class="op">├──&gt;</span> Vector <span class="comment">(LanceDB, 768-dim)</span>
+  <span class="op">├──&gt;</span> Vector <span class="comment">(SQLite BLOB, 768-dim)</span>
  <span class="op">├──&gt;</span> Annotation <span class="comment">(versioned, immutable)</span>
  <span class="op">└──&gt;</span> GraphEdge
         <span class="op">&darr;</span>
@ -1096,8 +1096,7 @@ Chunk <span class="comment">-- deterministic ID, 500-800 tokens</span>
 }</div>
        <h3 style="margin-top:1rem;">Storage</h3>
        <table class="data-table" style="font-size:0.8rem;">
-          <tr><td><strong>SQLite</strong></td><td>Metadata, graph, jobs (WAL mode)</td></tr>
-          <tr><td><strong>LanceDB</strong></td><td>Vectors + payload (embedded)</td></tr>
+          <tr><td><strong>SQLite</strong></td><td>Metadata, vectors (BLOB), graph, jobs (WAL mode)</td></tr>
          <tr><td><strong>Disk</strong></td><td>~/.knowledge-refinery/</td></tr>
        </table>
      </div>
@ -1165,7 +1164,7 @@ Chunk <span class="comment">-- deterministic ID, 500-800 tokens</span>
      <div class="card">
        <div class="icon violet">&#x1F4E6;</div>
        <h3>Multi-Workspace</h3>
-        <p>Each workspace gets its own data directory, daemon port (8742, 8743, ...), SQLite database, and LanceDB vector store. Independent lifecycle management.</p>
+        <p>Each workspace gets its own data directory, daemon port (8742, 8743, ...), and SQLite database (metadata + vectors + graph). Independent lifecycle management.</p>
      </div>
      <div class="card">
        <div class="icon blue">&#x1F5FA;</div>
@ -1213,18 +1212,18 @@ Chunk <span class="comment">-- deterministic ID, 500-800 tokens</span>
        <table class="data-table" style="margin-bottom:1.5rem;">
          <tr><td><strong>macOS</strong></td><td>Tahoe (26.x) on Apple Silicon</td></tr>
          <tr><td><strong>Xcode</strong></td><td>26.x or Command Line Tools</td></tr>
-          <tr><td><strong>Python</strong></td><td>3.12+ (system or python.org)</td></tr>
+          <tr><td><strong>Go</strong></td><td>1.22+ (from go.dev or Homebrew)</td></tr>
          <tr><td><strong>LM Studio</strong></td><td>From lmstudio.ai (free)</td></tr>
        </table>
        <h3>What the Installer Does</h3>
        <ol class="step-list">
          <li>
            <strong>Checks prerequisites</strong>
-            <p>Validates macOS version, architecture, Xcode tools, Swift, and Python version.</p>
+            <p>Validates macOS version, architecture, Xcode tools, Swift, and Go version.</p>
          </li>
          <li>
-            <strong>Creates Python venv</strong>
-            <p>Isolated <code>.venv</code> with all dependencies: FastAPI, LanceDB, PyMuPDF, tiktoken, NumPy.</p>
+            <strong>Builds Go daemon</strong>
+            <p>Compiles the Go daemon into a single 17MB binary with zero runtime dependencies.</p>
          </li>
          <li>
            <strong>Builds the .app bundle</strong>
@ -1260,10 +1259,7 @@ Knowledge Refinery.app/
      KnowledgeRefinery      <span class="comment"># launcher</span>
      KnowledgeRefinery-bin  <span class="comment"># Swift binary</span>
    Resources/
-      daemon/                <span class="comment"># Python source</span>
-        knowledge_refinery/
-        pyproject.toml
-        shared/
+      knowledge-refinery-daemon <span class="comment"># Go binary (17MB)</span>
      WebGPU/                <span class="comment"># 3D renderer</span>
        universe.html/js/wgsl</div>
      </div>
@ -1350,7 +1346,7 @@ Knowledge Refinery.app/
          <div style="color:var(--fg-subtle); padding-left:2.3rem;">&darr;</div>
          <div style="display:flex; align-items:center; gap:0.75rem;">
            <span style="background:#7c3aed; color:white; width:24px; height:24px; border-radius:50%; display:inline-flex; align-items:center; justify-content:center; font-size:0.7rem; font-weight:700; flex-shrink:0;">3</span>
-            LanceDB cosine similarity search
+            Brute-force cosine similarity search (in-memory vectors)
          </div>
          <div style="color:var(--fg-subtle); padding-left:2.3rem;">&darr;</div>
          <div style="display:flex; align-items:center; gap:0.75rem;">
@ -1449,20 +1445,20 @@ Knowledge Refinery.app/
        </table>
      </div>
      <div>
-        <h3>Backend (Python Daemon)</h3>
+        <h3>Backend (Go Daemon)</h3>
        <table class="data-table">
-          <tr><td><strong>Runtime</strong></td><td>Python 3.14.2 (venv isolated)</td></tr>
-          <tr><td><strong>Web Server</strong></td><td>FastAPI + Uvicorn</td></tr>
-          <tr><td><strong>Vector Store</strong></td><td>LanceDB (embedded, no server)</td></tr>
-          <tr><td><strong>Metadata</strong></td><td>SQLite 3 (WAL mode)</td></tr>
-          <tr><td><strong>LLM Client</strong></td><td>OpenAI SDK &rarr; LM Studio</td></tr>
-          <tr><td><strong>PDF</strong></td><td>PyMuPDF (fitz)</td></tr>
-          <tr><td><strong>Tokenizer</strong></td><td>tiktoken (cl100k_base)</td></tr>
-          <tr><td><strong>Math</strong></td><td>NumPy (clustering, kNN)</td></tr>
+          <tr><td><strong>Language</strong></td><td>Go 1.22+ (single 17MB binary)</td></tr>
+          <tr><td><strong>HTTP Router</strong></td><td>chi/v5 + CORS middleware</td></tr>
+          <tr><td><strong>Storage</strong></td><td>SQLite (metadata + vectors + graph, WAL mode)</td></tr>
+          <tr><td><strong>SQLite Driver</strong></td><td>modernc.org/sqlite (pure Go, no CGo)</td></tr>
+          <tr><td><strong>LLM Client</strong></td><td>net/http &rarr; LM Studio</td></tr>
+          <tr><td><strong>PDF</strong></td><td>pdftotext (poppler-utils)</td></tr>
+          <tr><td><strong>Tokenizer</strong></td><td>tiktoken-go (cl100k_base)</td></tr>
+          <tr><td><strong>Math</strong></td><td>Pure Go (k-means++, cosine sim)</td></tr>
        </table>
        <h3 style="margin-top:1.5rem;">LM Studio Models</h3>
        <table class="data-table">
-          <tr><td><strong>Chat</strong></td><td>google/gemma-3-4b</td></tr>
+          <tr><td><strong>Chat</strong></td><td>qwen3-4b-2507</td></tr>
          <tr><td><strong>Embeddings</strong></td><td>nomic-embed-text-v1.5 (768-dim)</td></tr>
        </table>
      </div>
@ -1481,15 +1477,15 @@ Knowledge Refinery.app/
      Clone, install, launch &mdash; from raw documents to searchable knowledge in minutes.
    </p>
    <div class="stats-row" style="justify-content:center; margin-bottom:2.5rem;">
-      <div class="stat"><div class="stat-num">30</div><div class="stat-label">Tests Passing</div></div>
+      <div class="stat"><div class="stat-num">97</div><div class="stat-label">Tests Passing</div></div>
      <div class="stat"><div class="stat-num">0</div><div class="stat-label">Compiler Warnings</div></div>
-      <div class="stat"><div class="stat-num">~55</div><div class="stat-label">Source Files</div></div>
+      <div class="stat"><div class="stat-num">~90</div><div class="stat-label">Source Files</div></div>
      <div class="stat"><div class="stat-num">100%</div><div class="stat-label">Local</div></div>
    </div>
    <div style="display:flex; gap:1rem; justify-content:center; flex-wrap:wrap; font-size:0.82rem;">
-      <div class="badge" style="background:var(--accent-bg); color:var(--accent);">&#x1F4C1; daemon/knowledge_refinery/</div>
+      <div class="badge" style="background:var(--accent-bg); color:var(--accent);">&#x1F4C1; daemon-go/</div>
      <div class="badge" style="background:#f5f3ff; color:#7c3aed;">&#x1F4F1; apps/macos/KnowledgeRefinery/</div>
-      <div class="badge" style="background:#ecfdf5; color:#059669;">&#x1F9EA; daemon/tests/</div>
+      <div class="badge" style="background:#ecfdf5; color:#059669;">&#x1F9EA; daemon-go/internal/*/</div>
      <div class="badge" style="background:#fffbeb; color:#d97706;">&#x1F6E0; scripts/install.sh</div>
    </div>
    <div style="margin-top:1.5rem;">
@ -1497,7 +1493,7 @@ Knowledge Refinery.app/
 <span class="fn">git</span> clone &lt;repo&gt; &amp;&amp; <span class="keyword">cd</span> LongLocalTimeHorizonInfoRetrieval &amp;&amp; <span class="fn">make</span> install</div>
    </div>
    <div style="margin-top:1.5rem; color:var(--fg-subtle); font-size:0.8rem;">
-      Built with SwiftUI, Python, FastAPI, LanceDB, WebGPU, and local LLMs
+      Built with SwiftUI, Go, chi, SQLite, WebGPU, and local LLMs
    </div>
  </div>
 </div>