JupyterManager/docs/LIFECYCLE_SPEC.md

# Jupyter Lifecycle Specification for Client Projects

**Version:** 1.0
**Purpose:** Ensure all Jupyter-based projects use aligned lifecycle management, preventing port conflicts, orphan processes, and cross-project interference.

---

## The Problem

Multiple Jupyter-based projects coexist on the same machine. Without alignment:
- Projects fight over port 8888
- Killing a terminal leaves orphan Jupyter processes
- Orphan processes block ports for other projects
- Jupyter config/runtime state leaks between projects

## Required Conventions

### 1. Isolated Jupyter Directories

Every project MUST isolate its Jupyter state into project-local directories. These directories prevent cross-project config leakage and make cleanup deterministic.

```bash
# These environment variables MUST be set before any jupyter command
export JUPYTER_CONFIG_DIR="$PROJECT_ROOT/.jupyter_config"
export JUPYTER_DATA_DIR="$PROJECT_ROOT/.jupyter_data"
export JUPYTER_RUNTIME_DIR="$PROJECT_ROOT/.jupyter_runtime"
export IPYTHONDIR="$PROJECT_ROOT/.ipython"
export MPLCONFIGDIR="$PROJECT_ROOT/.cache/matplotlib"
```

These directories MUST be listed in `.gitignore`.

### 2. Kernel Registration

Kernels MUST be installed into the venv (`--sys-prefix`), NOT into the user's global kernel directory (`--user`). This avoids kernel name collisions between projects.

```bash
python -m ipykernel install \
    --sys-prefix \
    --name "$PROJECT_NAME" \
    --display-name "$DISPLAY_NAME" \
    --env IPYTHONDIR "$PROJECT_ROOT/.ipython" \
    --env MPLCONFIGDIR "$PROJECT_ROOT/.cache/matplotlib"
```

### 3. app.sh Interface Contract

Every project MUST provide `scripts/app.sh` with at minimum these subcommands:

| Command | Behavior |
|---------|----------|
| `bootstrap` | Create venv, install deps, register kernel, create isolation dirs |
| `start` | Launch JupyterLab in background, write PID file, auto-find free port |
| `start --foreground` | Launch in foreground (Ctrl-C stops cleanly) |
| `start --no-open` | Launch without opening browser |
| `start --port PORT` | Use a specific port |
| `stop` | Send SIGTERM to PID, wait for graceful shutdown, clean PID file |
| `restart` | stop + start |
| `status` | Show venv, server, port, URL, orphan detection |
| `logs [-f]` | Show or follow log output |

### 4. PID File Location

PID files MUST be at `$PROJECT_ROOT/.logs/jupyter.pid` (preferred) or `$PROJECT_ROOT/.run/jupyter.pid` (acceptable). The cross-project manager (`jupyter-hub`) checks both locations.

### 5. Port Allocation

Projects MUST auto-detect free ports, not hardcode 8888:

```bash
local port=8888
while lsof -i :"$port" &>/dev/null; do
    port=$((port + 1))
    if (( port > 8899 )); then
        echo "No free port in range 8888-8899"
        exit 1
    fi
done
```

### 6. Background Process Management

When starting in background mode:
- Use `nohup` so the process survives shell close
- Write PID to the PID file immediately after `nohup ... &`
- Poll until the server responds (check port or API endpoint)
- Detect early exit (process dies during startup)

```bash
nohup "$JUPYTER" lab \
    --port="$port" \
    --no-browser \
    --ip=127.0.0.1 \
    --notebook-dir="$PROJECT_ROOT/notebooks" \
    > "$LOG_FILE" 2>&1 &

echo $! > "$PID_FILE"

# Wait for startup
while ! curl -s "http://localhost:$port/api" &>/dev/null; do
    sleep 0.5
    if ! kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then
        echo "Server died during startup"
        exit 1
    fi
done
```

### 7. Stop Protocol

```bash
pid=$(cat "$PID_FILE")

# 1. Send SIGTERM (graceful)
kill -TERM "$pid"

# 2. Wait up to 10 seconds for graceful shutdown
for i in $(seq 1 20); do
    kill -0 "$pid" 2>/dev/null || break
    sleep 0.5
done

# 3. If still alive, SIGKILL
if kill -0 "$pid" 2>/dev/null; then
    kill -KILL "$pid"
fi

# 4. Clean up
rm -f "$PID_FILE"
```

### 8. Orphan Detection

The `status` command SHOULD detect orphan Jupyter processes belonging to this project but not tracked by the PID file:

```bash
ps aux | grep "[j]upyter.*--notebook-dir.*$PROJECT_ROOT" | awk '{print $2}'
```

### 9. Stale Runtime Cleanup

On every `start` and `status`, clean up stale Jupyter runtime JSON files (where the PID is no longer alive):

```bash
for json_file in "$JUPYTER_RUNTIME_DIR"/jpserver-*.json; do
    pid=$(python3 -c "import json; print(json.load(open('$json_file')).get('pid',''))")
    if ! kill -0 "$pid" 2>/dev/null; then
        rm -f "$json_file"
    fi
done
```

### 10. .gitignore Entries

```gitignore
# Jupyter isolation directories
.jupyter_config/
.jupyter_data/
.jupyter_runtime/
.ipython/
.cache/

# Process management
.logs/
.run/
```