From d191344eb5469da1b6e2d32fdbcfdafe79269344 Mon Sep 17 00:00:00 2001 From: saymrwulf Date: Thu, 16 Apr 2026 09:07:43 +0200 Subject: [PATCH] Initial commit: jupyter-hub cross-project Jupyter manager CLI tool that discovers and manages JupyterLab instances across multiple projects. Prevents port conflicts, finds orphan processes, provides unified status and stop-all functionality. Includes: - bin/jupyter-hub: the CLI (status, ports, stop-all, orphans, which, config) - config/defaults.sh: default scan dirs and port range - docs/LIFECYCLE_SPEC.md: client project requirements - docs/AGENT_PROMPT.md: prompt for coding agents to align projects - install.sh: symlink installer to ~/bin - tests/test_jupyter_hub.sh: 15 functional tests Co-Authored-By: Claude Opus 4.6 --- .gitignore | 6 + README.md | 111 +++++++++ bin/jupyter-hub | 459 ++++++++++++++++++++++++++++++++++++++ config/defaults.sh | 18 ++ docs/AGENT_PROMPT.md | 43 ++++ docs/LIFECYCLE_SPEC.md | 166 ++++++++++++++ install.sh | 82 +++++++ tests/test_jupyter_hub.sh | 95 ++++++++ 8 files changed, 980 insertions(+) create mode 100644 .gitignore create mode 100644 README.md create mode 100755 bin/jupyter-hub create mode 100644 config/defaults.sh create mode 100644 docs/AGENT_PROMPT.md create mode 100644 docs/LIFECYCLE_SPEC.md create mode 100755 install.sh create mode 100755 tests/test_jupyter_hub.sh diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..3dcc88e --- /dev/null +++ b/.gitignore @@ -0,0 +1,6 @@ +# OS +.DS_Store + +# Editor +*.swp +*~ diff --git a/README.md b/README.md new file mode 100644 index 0000000..df6075d --- /dev/null +++ b/README.md @@ -0,0 +1,111 @@ +# JupyterManager + +Cross-project Jupyter instance manager. Discovers, monitors, and controls JupyterLab instances across multiple projects on the same machine. + +## The Problem + +You have multiple projects with Jupyter notebooks. Each runs its own JupyterLab server. Without coordination: + +- Projects fight over port 8888 +- Closing a terminal leaves orphan Jupyter processes +- Orphans block ports for other projects +- You can't tell what's running or which project owns which port + +## What JupyterManager Does + +**`jupyter-hub`** is a single command that manages everything: + +```bash +jupyter-hub status # Show all projects (running/stopped) +jupyter-hub ports # Port allocation map (8888-8899) +jupyter-hub stop-all # Stop all Jupyter instances +jupyter-hub orphans # Find untracked processes +jupyter-hub kill-orphans # Kill them +jupyter-hub which 8889 # Which project owns this port? +jupyter-hub config # Show configuration +``` + +It discovers projects automatically by scanning configured directories for the `scripts/app.sh` convention. + +## Installation + +```bash +git clone https://github.com/saymrwulf/JupyterManager.git +cd JupyterManager +bash install.sh +``` + +This creates a symlink at `~/bin/jupyter-hub` pointing to the repo's `bin/jupyter-hub`. Updates are just `git pull`. + +Ensure `~/bin` is in your PATH: + +```bash +# Add to ~/.zshrc or ~/.bashrc +export PATH="$HOME/bin:$PATH" +``` + +## Configuration + +Configuration lives at `~/.config/jupyter-hub/config.sh` (created by `install.sh`): + +```bash +# Port range to scan +PORT_RANGE_START=8888 +PORT_RANGE_END=8899 + +# Directories to scan for Jupyter projects +SCAN_DIRS=( + "$HOME/GitClone/ClaudeCodeProjects" + "$HOME/GitClone/CodexProjects" + "$HOME/Projects" +) +``` + +A project is recognized if it has `scripts/app.sh` AND either a `notebooks/` directory or `jupyter` in its `pyproject.toml`. + +## Client Project Requirements + +For a project to be managed by `jupyter-hub`, it must follow the **Lifecycle Specification** documented in [`docs/LIFECYCLE_SPEC.md`](docs/LIFECYCLE_SPEC.md). The key requirements: + +1. `scripts/app.sh` with: `bootstrap`, `start`, `stop`, `restart`, `status`, `logs` +2. Isolated Jupyter directories under the project root (not system-global) +3. Kernel installed with `--sys-prefix` (not `--user`) +4. Auto port allocation (scan 8888-8899 for free port) +5. PID file at `.logs/jupyter.pid` +6. Background + foreground modes +7. Graceful stop protocol (SIGTERM, wait, SIGKILL fallback) + +### Current Client Projects + +| Project | Location | Status | +|---------|----------|--------| +| autoresearch-quantum | `ClaudeCodeProjects/` | Fully compliant | +| QuantumLearning | `CodexProjects/` | Detected, partial compliance | +| NTT-learning | `CodexProjects/` | Detected, partial compliance | + +## For Coding Agents + +When instructing an AI coding agent to align a project with this specification, use the prompt in [`docs/AGENT_PROMPT.md`](docs/AGENT_PROMPT.md). + +## Project Structure + +``` +JupyterManager/ +├── bin/ +│ └── jupyter-hub The CLI tool +├── config/ +│ └── defaults.sh Default scan dirs, port range +├── docs/ +│ ├── LIFECYCLE_SPEC.md Client project specification +│ └── AGENT_PROMPT.md Prompt for coding agents +├── tests/ +│ └── test_jupyter_hub.sh Functional tests +├── install.sh Symlink installer +└── README.md +``` + +## Running Tests + +```bash +bash tests/test_jupyter_hub.sh +``` diff --git a/bin/jupyter-hub b/bin/jupyter-hub new file mode 100755 index 0000000..9e71595 --- /dev/null +++ b/bin/jupyter-hub @@ -0,0 +1,459 @@ +#!/usr/bin/env bash +# ────────────────────────────────────────────────────────────────────── +# jupyter-hub — Cross-project Jupyter instance manager +# +# Discovers and manages ALL JupyterLab instances across your projects. +# Prevents port conflicts, finds orphan processes, provides unified +# status and stop-all functionality. +# +# Part of: https://github.com/saymrwulf/JupyterManager +# +# Usage: +# jupyter-hub status Show all running Jupyter instances +# jupyter-hub stop-all Stop all running Jupyter instances +# jupyter-hub ports Show port allocation across 8888-8899 +# jupyter-hub orphans Find orphan Jupyter processes +# jupyter-hub kill-orphans Kill orphan Jupyter processes +# jupyter-hub which PORT Show which project owns a port +# jupyter-hub config Show current configuration +# ────────────────────────────────────────────────────────────────────── +set -euo pipefail + +JHUB_VERSION="1.0.0" +JHUB_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +JHUB_CONFIG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/jupyter-hub" +JHUB_CONFIG_FILE="$JHUB_CONFIG_DIR/config.sh" + +# ── Colours ─────────────────────────────────────────────────────────── +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +BOLD='\033[1m' +DIM='\033[2m' +NC='\033[0m' + +# ── Default configuration ──────────────────────────────────────────── +# Override in $JHUB_CONFIG_FILE +PORT_RANGE_START=8888 +PORT_RANGE_END=8899 +SCAN_DIRS=() + +# ── Load configuration ─────────────────────────────────────────────── + +_load_config() { + # Load project-shipped defaults + if [[ -f "$JHUB_ROOT/config/defaults.sh" ]]; then + source "$JHUB_ROOT/config/defaults.sh" + fi + + # Load user overrides + if [[ -f "$JHUB_CONFIG_FILE" ]]; then + source "$JHUB_CONFIG_FILE" + fi + + # Fallback if no scan dirs configured + if [[ ${#SCAN_DIRS[@]} -eq 0 ]]; then + SCAN_DIRS=( + "$HOME/GitClone/ClaudeCodeProjects" + "$HOME/GitClone/CodexProjects" + ) + fi +} + +# ── Discovery ───────────────────────────────────────────────────────── + +_discover_projects() { + local projects=() + for scan_dir in "${SCAN_DIRS[@]}"; do + if [[ -d "$scan_dir" ]]; then + while IFS= read -r -d '' app_sh; do + local project_dir + project_dir="$(dirname "$(dirname "$app_sh")")" + # Verify it's a Jupyter-capable project + if [[ -d "$project_dir/notebooks" ]] || grep -q "jupyter" "$project_dir/pyproject.toml" 2>/dev/null; then + projects+=("$project_dir") + fi + done < <(find "$scan_dir" -maxdepth 3 -path "*/scripts/app.sh" -print0 2>/dev/null) + fi + done + printf '%s\n' "${projects[@]}" | sort -u +} + +_project_name() { + basename "$1" +} + +_pid_alive() { + kill -0 "$1" 2>/dev/null +} + +_find_server_info() { + local project_dir="$1" + + # Strategy 1: .logs/jupyter.pid + local pid_file="$project_dir/.logs/jupyter.pid" + if [[ -f "$pid_file" ]]; then + local pid + pid=$(cat "$pid_file") + if [[ -n "$pid" ]] && _pid_alive "$pid"; then + echo "pid=$pid source=pid_file" + return 0 + fi + fi + + # Strategy 2: .run/jupyter.pid + pid_file="$project_dir/.run/jupyter.pid" + if [[ -f "$pid_file" ]]; then + local pid + pid=$(cat "$pid_file") + if [[ -n "$pid" ]] && _pid_alive "$pid"; then + echo "pid=$pid source=run_pid" + return 0 + fi + fi + + # Strategy 3: Jupyter runtime JSON files + for runtime_dir in "$project_dir/.jupyter_runtime" "$project_dir/.run"; do + if [[ -d "$runtime_dir" ]]; then + for json_file in "$runtime_dir"/jpserver-*.json; do + if [[ -f "$json_file" ]]; then + local pid port token + pid=$(python3 -c "import json; d=json.load(open('$json_file')); print(d.get('pid',''))" 2>/dev/null || true) + port=$(python3 -c "import json; d=json.load(open('$json_file')); print(d.get('port',''))" 2>/dev/null || true) + token=$(python3 -c "import json; d=json.load(open('$json_file')); print(d.get('token',''))" 2>/dev/null || true) + if [[ -n "$pid" ]] && _pid_alive "$pid"; then + echo "pid=$pid port=$port token=$token source=runtime_json" + return 0 + fi + fi + done + fi + done + + return 1 +} + +_pid_port() { + local pid="$1" + lsof -Pan -p "$pid" -i TCP -sTCP:LISTEN 2>/dev/null | awk 'NR>1 { + split($9, a, ":"); print a[length(a)] + }' | head -1 +} + +# ── Commands ────────────────────────────────────────────────────────── + +cmd_status() { + echo "" + echo -e " ${BOLD}Jupyter Hub${NC} v${JHUB_VERSION}" + echo -e " ${DIM}$(date '+%Y-%m-%d %H:%M:%S')${NC}" + echo "" + + local found_any=false + + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + local name + name=$(_project_name "$project_dir") + + local info + if info=$(_find_server_info "$project_dir"); then + found_any=true + local pid port + pid=$(echo "$info" | grep -o 'pid=[0-9]*' | cut -d= -f2) + port=$(_pid_port "$pid" 2>/dev/null || echo "?") + echo -e " ${GREEN}●${NC} ${BOLD}$name${NC}" + echo -e " PID: $pid Port: $port" + echo -e " Dir: ${DIM}$project_dir${NC}" + if [[ "$port" != "?" ]]; then + echo -e " URL: http://localhost:$port/lab" + fi + else + echo -e " ${DIM}○ $name${NC} ${DIM}(stopped)${NC}" + fi + done < <(_discover_projects) + + if ! $found_any; then + echo -e " ${DIM}No running Jupyter instances found.${NC}" + fi + + echo "" +} + +cmd_ports() { + echo "" + echo -e " ${BOLD}Port Allocation (${PORT_RANGE_START}-${PORT_RANGE_END})${NC}" + echo "" + + for port in $(seq "$PORT_RANGE_START" "$PORT_RANGE_END"); do + local pid + pid=$(lsof -ti :"$port" 2>/dev/null | head -1 || true) + if [[ -n "$pid" ]]; then + local cmd_line project_name="unknown" + cmd_line=$(ps -p "$pid" -o args= 2>/dev/null || true) + if [[ "$cmd_line" =~ notebook-dir[=\ ]*([^ ]*) ]]; then + project_name=$(basename "$(dirname "${BASH_REMATCH[1]}")") + elif [[ "$cmd_line" =~ root_dir[=\ ]*([^ ]*) ]]; then + project_name=$(basename "${BASH_REMATCH[1]}") + fi + echo -e " ${RED}$port${NC} pid:$pid ${BOLD}$project_name${NC}" + echo -e " ${DIM}$(echo "$cmd_line" | head -c 100)${NC}" + else + echo -e " ${GREEN}$port${NC} ${DIM}free${NC}" + fi + done + echo "" +} + +cmd_stop_all() { + echo "" + local stopped=0 + + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + local name + name=$(_project_name "$project_dir") + + local info + if info=$(_find_server_info "$project_dir"); then + local pid + pid=$(echo "$info" | grep -o 'pid=[0-9]*' | cut -d= -f2) + + if [[ -x "$project_dir/scripts/app.sh" ]]; then + echo -e " Stopping ${BOLD}$name${NC} via app.sh..." + (cd "$project_dir" && bash scripts/app.sh stop 2>/dev/null) || true + else + echo -e " Stopping ${BOLD}$name${NC} (pid $pid)..." + kill -TERM "$pid" 2>/dev/null || true + fi + stopped=$((stopped + 1)) + fi + done < <(_discover_projects) + + # Also kill any orphans not claimed by known projects + local orphan_count=0 + while IFS= read -r pid; do + [[ -z "$pid" ]] && continue + # Skip PIDs we already stopped via app.sh + local already_stopped=false + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + for pf in "$project_dir/.logs/jupyter.pid" "$project_dir/.run/jupyter.pid"; do + if [[ -f "$pf" ]] && [[ "$(cat "$pf" 2>/dev/null)" == "$pid" ]]; then + already_stopped=true + break 2 + fi + done + done < <(_discover_projects) + + if ! $already_stopped && _pid_alive "$pid"; then + echo -e " Stopping orphan pid $pid..." + kill -TERM "$pid" 2>/dev/null || true + orphan_count=$((orphan_count + 1)) + fi + done < <(pgrep -f "jupyter.*lab" 2>/dev/null || true) + + local total=$((stopped + orphan_count)) + if (( total > 0 )); then + echo "" + echo -e " ${GREEN}Stopped $total instance(s).${NC}" + else + echo -e " ${DIM}No running instances to stop.${NC}" + fi + echo "" +} + +cmd_orphans() { + echo "" + echo -e " ${BOLD}Orphan Jupyter Processes${NC}" + echo -e " ${DIM}(Jupyter processes not tracked by any project's PID file)${NC}" + echo "" + + local found=false + while IFS= read -r line; do + [[ -z "$line" ]] && continue + local pid + pid=$(echo "$line" | awk '{print $2}') + + local claimed=false + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + for pid_file in "$project_dir/.logs/jupyter.pid" "$project_dir/.run/jupyter.pid"; do + if [[ -f "$pid_file" ]] && [[ "$(cat "$pid_file")" == "$pid" ]]; then + claimed=true + break 2 + fi + done + for json_file in "$project_dir/.jupyter_runtime"/jpserver-*.json; do + if [[ -f "$json_file" ]]; then + local json_pid + json_pid=$(python3 -c "import json; print(json.load(open('$json_file')).get('pid',''))" 2>/dev/null || true) + if [[ "$json_pid" == "$pid" ]]; then + claimed=true + break 2 + fi + fi + done + done < <(_discover_projects) + + if ! $claimed; then + found=true + local port cmd_short + port=$(_pid_port "$pid" 2>/dev/null || echo "?") + cmd_short=$(ps -p "$pid" -o args= 2>/dev/null | head -c 100 || echo "?") + echo -e " ${YELLOW}!${NC} PID $pid Port: $port" + echo -e " ${DIM}$cmd_short${NC}" + fi + done < <(ps aux | grep "[j]upyter.*lab" || true) + + if ! $found; then + echo -e " ${GREEN}No orphan processes found.${NC}" + else + echo "" + echo " To kill orphans: jupyter-hub kill-orphans" + fi + echo "" +} + +cmd_kill_orphans() { + local orphans=() + while IFS= read -r line; do + [[ -z "$line" ]] && continue + local pid + pid=$(echo "$line" | awk '{print $2}') + + local claimed=false + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + for pid_file in "$project_dir/.logs/jupyter.pid" "$project_dir/.run/jupyter.pid"; do + if [[ -f "$pid_file" ]] && [[ "$(cat "$pid_file")" == "$pid" ]]; then + claimed=true + break 2 + fi + done + done < <(_discover_projects) + + if ! $claimed; then + orphans+=("$pid") + fi + done < <(ps aux | grep "[j]upyter.*lab" || true) + + if [[ ${#orphans[@]} -eq 0 ]]; then + echo "No orphan Jupyter processes found." + return 0 + fi + + for pid in "${orphans[@]}"; do + echo "Killing orphan pid $pid..." + kill -TERM "$pid" 2>/dev/null || true + done + echo "Killed ${#orphans[@]} orphan(s)." +} + +cmd_which() { + local port="${1:-}" + if [[ -z "$port" ]]; then + echo "Usage: jupyter-hub which PORT" + exit 1 + fi + + local pid + pid=$(lsof -ti :"$port" 2>/dev/null | head -1 || true) + if [[ -z "$pid" ]]; then + echo "Port $port is free." + return 0 + fi + + echo "Port $port is used by pid $pid" + local cmd_line + cmd_line=$(ps -p "$pid" -o args= 2>/dev/null || echo "unknown") + echo "Command: $cmd_line" + + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + local info + if info=$(_find_server_info "$project_dir" 2>/dev/null); then + local proj_pid + proj_pid=$(echo "$info" | grep -o 'pid=[0-9]*' | cut -d= -f2) + if [[ "$proj_pid" == "$pid" ]]; then + echo "Project: $(_project_name "$project_dir")" + echo "Path: $project_dir" + return 0 + fi + fi + done < <(_discover_projects) + + echo "Project: (not matched to any known project)" +} + +cmd_config() { + echo "" + echo -e " ${BOLD}Jupyter Hub Configuration${NC}" + echo "" + echo " Install root: $JHUB_ROOT" + echo " Config dir: $JHUB_CONFIG_DIR" + echo " Config file: $JHUB_CONFIG_FILE" + if [[ -f "$JHUB_CONFIG_FILE" ]]; then + echo " Config status: loaded" + else + echo " Config status: using defaults" + fi + echo " Port range: ${PORT_RANGE_START}-${PORT_RANGE_END}" + echo "" + echo " Scan directories:" + for d in "${SCAN_DIRS[@]}"; do + if [[ -d "$d" ]]; then + echo -e " ${GREEN}✓${NC} $d" + else + echo -e " ${RED}✗${NC} $d (not found)" + fi + done + echo "" + echo " Discovered projects:" + while IFS= read -r project_dir; do + [[ -z "$project_dir" ]] && continue + echo " $(_project_name "$project_dir") → $project_dir" + done < <(_discover_projects) + echo "" +} + +# ── Main ────────────────────────────────────────────────────────────── + +_load_config + +case "${1:-help}" in + status) cmd_status ;; + ports) cmd_ports ;; + stop-all) cmd_stop_all ;; + orphans) cmd_orphans ;; + kill-orphans) cmd_kill_orphans ;; + which) cmd_which "${2:-}" ;; + config) cmd_config ;; + version|--version|-v) + echo "jupyter-hub v${JHUB_VERSION}" + ;; + help|--help|-h) + echo "" + echo -e " ${BOLD}jupyter-hub${NC} v${JHUB_VERSION} — cross-project Jupyter manager" + echo "" + echo " Commands:" + echo " status Show all running Jupyter instances" + echo " stop-all Stop all running instances" + echo " ports Show port allocation (${PORT_RANGE_START}-${PORT_RANGE_END})" + echo " orphans Find orphan Jupyter processes" + echo " kill-orphans Kill orphan processes" + echo " which PORT Show which project owns a port" + echo " config Show current configuration" + echo " version Show version" + echo "" + echo " Configuration:" + echo " $JHUB_CONFIG_FILE" + echo "" + ;; + *) + echo "Unknown command: $1" + echo "Run 'jupyter-hub help' for usage." + exit 1 + ;; +esac diff --git a/config/defaults.sh b/config/defaults.sh new file mode 100644 index 0000000..55a8166 --- /dev/null +++ b/config/defaults.sh @@ -0,0 +1,18 @@ +# ────────────────────────────────────────────────────────────────────── +# jupyter-hub default configuration +# +# Override these in ~/.config/jupyter-hub/config.sh +# ────────────────────────────────────────────────────────────────────── + +# Port range to scan for Jupyter instances +PORT_RANGE_START=8888 +PORT_RANGE_END=8899 + +# Directories to scan for projects with scripts/app.sh +# A project is recognized if it has: +# 1. scripts/app.sh AND +# 2. a notebooks/ directory OR jupyter in pyproject.toml +SCAN_DIRS=( + "$HOME/GitClone/ClaudeCodeProjects" + "$HOME/GitClone/CodexProjects" +) diff --git a/docs/AGENT_PROMPT.md b/docs/AGENT_PROMPT.md new file mode 100644 index 0000000..ae8a298 --- /dev/null +++ b/docs/AGENT_PROMPT.md @@ -0,0 +1,43 @@ +# Agent Prompt for Jupyter Lifecycle Alignment + +Use this prompt when instructing a coding agent (Claude, Codex, etc.) to add or fix Jupyter lifecycle management in a client project. + +--- + +## The Prompt + +``` +This project needs Jupyter lifecycle management aligned with the JupyterManager +cross-project specification. + +Reference implementation: https://github.com/saymrwulf/JupyterManager +Spec: docs/LIFECYCLE_SPEC.md in that repo. + +Key requirements: +1. scripts/app.sh with: bootstrap, start, stop, restart, status, logs +2. Isolated Jupyter dirs (JUPYTER_CONFIG_DIR, DATA_DIR, RUNTIME_DIR, IPYTHONDIR) + all under the project root — set as env vars before any jupyter command +3. Kernel installed with --sys-prefix (not --user) +4. Auto port allocation (scan 8888-8899 for free port) +5. PID file at .logs/jupyter.pid +6. Background mode with nohup, foreground mode with exec +7. Graceful stop (SIGTERM → wait → SIGKILL fallback) +8. Orphan detection in status command +9. Stale runtime JSON cleanup on start/status +10. All isolation dirs in .gitignore + +The cross-project manager (jupyter-hub) expects: +- PID files at .logs/jupyter.pid or .run/jupyter.pid +- Runtime JSON at .jupyter_runtime/jpserver-*.json +- scripts/app.sh stop to be callable from any working directory + +Do NOT: +- Use --user for kernel installation +- Hardcode port 8888 +- Skip the startup wait loop +- Leave out foreground mode +- Forget .gitignore entries for isolation dirs + +A reference app.sh implementation exists at: + https://github.com/saymrwulf/autoresearch-quantum/blob/master/scripts/app.sh +``` diff --git a/docs/LIFECYCLE_SPEC.md b/docs/LIFECYCLE_SPEC.md new file mode 100644 index 0000000..ce94670 --- /dev/null +++ b/docs/LIFECYCLE_SPEC.md @@ -0,0 +1,166 @@ +# Jupyter Lifecycle Specification for Client Projects + +**Version:** 1.0 +**Purpose:** Ensure all Jupyter-based projects use aligned lifecycle management, preventing port conflicts, orphan processes, and cross-project interference. + +--- + +## The Problem + +Multiple Jupyter-based projects coexist on the same machine. Without alignment: +- Projects fight over port 8888 +- Killing a terminal leaves orphan Jupyter processes +- Orphan processes block ports for other projects +- Jupyter config/runtime state leaks between projects + +## Required Conventions + +### 1. Isolated Jupyter Directories + +Every project MUST isolate its Jupyter state into project-local directories. These directories prevent cross-project config leakage and make cleanup deterministic. + +```bash +# These environment variables MUST be set before any jupyter command +export JUPYTER_CONFIG_DIR="$PROJECT_ROOT/.jupyter_config" +export JUPYTER_DATA_DIR="$PROJECT_ROOT/.jupyter_data" +export JUPYTER_RUNTIME_DIR="$PROJECT_ROOT/.jupyter_runtime" +export IPYTHONDIR="$PROJECT_ROOT/.ipython" +export MPLCONFIGDIR="$PROJECT_ROOT/.cache/matplotlib" +``` + +These directories MUST be listed in `.gitignore`. + +### 2. Kernel Registration + +Kernels MUST be installed into the venv (`--sys-prefix`), NOT into the user's global kernel directory (`--user`). This avoids kernel name collisions between projects. + +```bash +python -m ipykernel install \ + --sys-prefix \ + --name "$PROJECT_NAME" \ + --display-name "$DISPLAY_NAME" \ + --env IPYTHONDIR "$PROJECT_ROOT/.ipython" \ + --env MPLCONFIGDIR "$PROJECT_ROOT/.cache/matplotlib" +``` + +### 3. app.sh Interface Contract + +Every project MUST provide `scripts/app.sh` with at minimum these subcommands: + +| Command | Behavior | +|---------|----------| +| `bootstrap` | Create venv, install deps, register kernel, create isolation dirs | +| `start` | Launch JupyterLab in background, write PID file, auto-find free port | +| `start --foreground` | Launch in foreground (Ctrl-C stops cleanly) | +| `start --no-open` | Launch without opening browser | +| `start --port PORT` | Use a specific port | +| `stop` | Send SIGTERM to PID, wait for graceful shutdown, clean PID file | +| `restart` | stop + start | +| `status` | Show venv, server, port, URL, orphan detection | +| `logs [-f]` | Show or follow log output | + +### 4. PID File Location + +PID files MUST be at `$PROJECT_ROOT/.logs/jupyter.pid` (preferred) or `$PROJECT_ROOT/.run/jupyter.pid` (acceptable). The cross-project manager (`jupyter-hub`) checks both locations. + +### 5. Port Allocation + +Projects MUST auto-detect free ports, not hardcode 8888: + +```bash +local port=8888 +while lsof -i :"$port" &>/dev/null; do + port=$((port + 1)) + if (( port > 8899 )); then + echo "No free port in range 8888-8899" + exit 1 + fi +done +``` + +### 6. Background Process Management + +When starting in background mode: +- Use `nohup` so the process survives shell close +- Write PID to the PID file immediately after `nohup ... &` +- Poll until the server responds (check port or API endpoint) +- Detect early exit (process dies during startup) + +```bash +nohup "$JUPYTER" lab \ + --port="$port" \ + --no-browser \ + --ip=127.0.0.1 \ + --notebook-dir="$PROJECT_ROOT/notebooks" \ + > "$LOG_FILE" 2>&1 & + +echo $! > "$PID_FILE" + +# Wait for startup +while ! curl -s "http://localhost:$port/api" &>/dev/null; do + sleep 0.5 + if ! kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then + echo "Server died during startup" + exit 1 + fi +done +``` + +### 7. Stop Protocol + +```bash +pid=$(cat "$PID_FILE") + +# 1. Send SIGTERM (graceful) +kill -TERM "$pid" + +# 2. Wait up to 10 seconds for graceful shutdown +for i in $(seq 1 20); do + kill -0 "$pid" 2>/dev/null || break + sleep 0.5 +done + +# 3. If still alive, SIGKILL +if kill -0 "$pid" 2>/dev/null; then + kill -KILL "$pid" +fi + +# 4. Clean up +rm -f "$PID_FILE" +``` + +### 8. Orphan Detection + +The `status` command SHOULD detect orphan Jupyter processes belonging to this project but not tracked by the PID file: + +```bash +ps aux | grep "[j]upyter.*--notebook-dir.*$PROJECT_ROOT" | awk '{print $2}' +``` + +### 9. Stale Runtime Cleanup + +On every `start` and `status`, clean up stale Jupyter runtime JSON files (where the PID is no longer alive): + +```bash +for json_file in "$JUPYTER_RUNTIME_DIR"/jpserver-*.json; do + pid=$(python3 -c "import json; print(json.load(open('$json_file')).get('pid',''))") + if ! kill -0 "$pid" 2>/dev/null; then + rm -f "$json_file" + fi +done +``` + +### 10. .gitignore Entries + +```gitignore +# Jupyter isolation directories +.jupyter_config/ +.jupyter_data/ +.jupyter_runtime/ +.ipython/ +.cache/ + +# Process management +.logs/ +.run/ +``` diff --git a/install.sh b/install.sh new file mode 100755 index 0000000..6491d17 --- /dev/null +++ b/install.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +# ────────────────────────────────────────────────────────────────────── +# install.sh — Install jupyter-hub to ~/bin via symlink +# +# Usage: +# bash install.sh Install (symlink to ~/bin/jupyter-hub) +# bash install.sh --remove Remove the symlink +# ────────────────────────────────────────────────────────────────────── +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +TARGET="$HOME/bin/jupyter-hub" +SOURCE="$SCRIPT_DIR/bin/jupyter-hub" +CONFIG_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/jupyter-hub" + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BOLD='\033[1m' +NC='\033[0m' + +if [[ "${1:-}" == "--remove" ]]; then + if [[ -L "$TARGET" ]]; then + rm "$TARGET" + echo -e "${GREEN}Removed${NC} $TARGET" + elif [[ -f "$TARGET" ]]; then + echo -e "${YELLOW}Warning:${NC} $TARGET is not a symlink (may be a standalone copy)" + echo " Remove manually: rm $TARGET" + else + echo "Nothing to remove — $TARGET does not exist." + fi + exit 0 +fi + +# Create ~/bin if needed +mkdir -p "$(dirname "$TARGET")" + +# Remove old standalone copy if present (from before this project existed) +if [[ -f "$TARGET" && ! -L "$TARGET" ]]; then + echo -e "${YELLOW}Replacing standalone ~/bin/jupyter-hub with symlink${NC}" + rm "$TARGET" +fi + +# Create symlink +ln -sf "$SOURCE" "$TARGET" +echo -e "${GREEN}Installed${NC} $TARGET → $SOURCE" + +# Create default config if not present +if [[ ! -f "$CONFIG_DIR/config.sh" ]]; then + mkdir -p "$CONFIG_DIR" + cat > "$CONFIG_DIR/config.sh" << 'CONF' +# ────────────────────────────────────────────────────────────────────── +# jupyter-hub user configuration +# +# Override defaults here. Sourced by jupyter-hub on startup. +# ────────────────────────────────────────────────────────────────────── + +# Port range to scan +# PORT_RANGE_START=8888 +# PORT_RANGE_END=8899 + +# Directories to scan for Jupyter projects +# Uncomment and edit: +# SCAN_DIRS=( +# "$HOME/GitClone/ClaudeCodeProjects" +# "$HOME/GitClone/CodexProjects" +# "$HOME/Projects" +# ) +CONF + echo -e "${GREEN}Created${NC} $CONFIG_DIR/config.sh" +fi + +# Check PATH +if ! echo "$PATH" | tr ':' '\n' | grep -q "$HOME/bin"; then + echo "" + echo -e "${YELLOW}Note:${NC} ~/bin is not in your PATH." + echo " Add to your shell profile (~/.zshrc or ~/.bashrc):" + echo ' export PATH="$HOME/bin:$PATH"' +fi + +echo "" +echo -e "${BOLD}Done.${NC} Run: jupyter-hub status" diff --git a/tests/test_jupyter_hub.sh b/tests/test_jupyter_hub.sh new file mode 100755 index 0000000..1123595 --- /dev/null +++ b/tests/test_jupyter_hub.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +# ────────────────────────────────────────────────────────────────────── +# test_jupyter_hub.sh — Tests for jupyter-hub +# +# Usage: bash tests/test_jupyter_hub.sh +# ────────────────────────────────────────────────────────────────────── +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +JHUB="$SCRIPT_DIR/bin/jupyter-hub" + +passed=0 +failed=0 + +pass() { echo " ✓ $1"; passed=$((passed + 1)); } +fail() { echo " ✗ $1: $2"; failed=$((failed + 1)); } + +assert_exit_0() { + local desc="$1"; shift + if "$@" >/dev/null 2>&1; then + pass "$desc" + else + fail "$desc" "exited with $?" + fi +} + +assert_output_contains() { + local desc="$1" + local expected="$2" + shift 2 + local output + output=$("$@" 2>&1) || true + if echo "$output" | grep -q "$expected"; then + pass "$desc" + else + fail "$desc" "output did not contain '$expected'" + fi +} + +echo "" +echo "Testing jupyter-hub" +echo "" + +# ── Basic invocation ────────────────────────────────────────────────── +echo "Basic invocation:" +assert_exit_0 "help exits 0" "$JHUB" help +assert_exit_0 "version exits 0" "$JHUB" version +assert_exit_0 "config exits 0" "$JHUB" config +assert_output_contains "version shows version string" "jupyter-hub v" "$JHUB" version +assert_output_contains "help shows Commands section" "Commands:" "$JHUB" help + +# ── Status command ──────────────────────────────────────────────────── +echo "" +echo "Status command:" +assert_exit_0 "status exits 0" "$JHUB" status +assert_output_contains "status shows header" "Jupyter Hub" "$JHUB" status + +# ── Ports command ───────────────────────────────────────────────────── +echo "" +echo "Ports command:" +assert_exit_0 "ports exits 0" "$JHUB" ports +assert_output_contains "ports shows 8888" "8888" "$JHUB" ports + +# ── Config command ──────────────────────────────────────────────────── +echo "" +echo "Config command:" +assert_output_contains "config shows install root" "Install root" "$JHUB" config +assert_output_contains "config shows port range" "Port range" "$JHUB" config +assert_output_contains "config shows scan dirs" "Scan directories" "$JHUB" config + +# ── Which command ───────────────────────────────────────────────────── +echo "" +echo "Which command:" +# Use a port that's almost certainly free +assert_output_contains "which on free port says free" "free" "$JHUB" which 19999 + +# ── Orphans command ─────────────────────────────────────────────────── +echo "" +echo "Orphans command:" +assert_exit_0 "orphans exits 0" "$JHUB" orphans + +# ── Unknown command ─────────────────────────────────────────────────── +echo "" +echo "Error handling:" +assert_output_contains "unknown command prints error" "Unknown command" "$JHUB" nonexistent + +# ── Summary ─────────────────────────────────────────────────────────── +echo "" +total=$((passed + failed)) +if (( failed == 0 )); then + echo "All $total tests passed." +else + echo "$passed/$total passed, $failed FAILED." + exit 1 +fi