JupyterManager/docs/LIFECYCLE_SPEC.md

Jupyter Lifecycle Specification for Client Projects

Version: 1.0 Purpose: Ensure all Jupyter-based projects use aligned lifecycle management, preventing port conflicts, orphan processes, and cross-project interference.

---

The Problem

Multiple Jupyter-based projects coexist on the same machine. Without alignment:

• Projects fight over port 8888
• Killing a terminal leaves orphan Jupyter processes
• Orphan processes block ports for other projects
• Jupyter config/runtime state leaks between projects

Required Conventions

1. Isolated Jupyter Directories

Every project MUST isolate its Jupyter state into project-local directories. These directories prevent cross-project config leakage and make cleanup deterministic.

# These environment variables MUST be set before any jupyter command
export JUPYTER_CONFIG_DIR="$PROJECT_ROOT/.jupyter_config"
export JUPYTER_DATA_DIR="$PROJECT_ROOT/.jupyter_data"
export JUPYTER_RUNTIME_DIR="$PROJECT_ROOT/.jupyter_runtime"
export IPYTHONDIR="$PROJECT_ROOT/.ipython"
export MPLCONFIGDIR="$PROJECT_ROOT/.cache/matplotlib"

These directories MUST be listed in .gitignore.

2. Kernel Registration

Kernels MUST be installed into the venv (--sys-prefix), NOT into the user's global kernel directory (--user). This avoids kernel name collisions between projects.

python -m ipykernel install \
    --sys-prefix \
    --name "$PROJECT_NAME" \
    --display-name "$DISPLAY_NAME" \
    --env IPYTHONDIR "$PROJECT_ROOT/.ipython" \
    --env MPLCONFIGDIR "$PROJECT_ROOT/.cache/matplotlib"

3. app.sh Interface Contract

Every project MUST provide scripts/app.sh with at minimum these subcommands:

Command	Behavior
bootstrap	Create venv, install deps, register kernel, create isolation dirs
start	Launch JupyterLab in background, write PID file, auto-find free port
start --foreground	Launch in foreground (Ctrl-C stops cleanly)
start --no-open	Launch without opening browser
start --port PORT	Use a specific port
stop	Send SIGTERM to PID, wait for graceful shutdown, clean PID file
restart	stop + start
status	Show venv, server, port, URL, orphan detection
logs [-f]	Show or follow log output

4. PID File Location

PID files MUST be at $PROJECT_ROOT/.logs/jupyter.pid (preferred) or $PROJECT_ROOT/.run/jupyter.pid (acceptable). The cross-project manager (jupyter-hub) checks both locations.

5. Port Allocation

Projects MUST auto-detect free ports, not hardcode 8888:

local port=8888
while lsof -i :"$port" &>/dev/null; do
    port=$((port + 1))
    if (( port > 8899 )); then
        echo "No free port in range 8888-8899"
        exit 1
    fi
done

6. Background Process Management

When starting in background mode:

• Use nohup so the process survives shell close
• Write PID to the PID file immediately after nohup ... &
• Poll until the server responds (check port or API endpoint)
• Detect early exit (process dies during startup)

nohup "$JUPYTER" lab \
    --port="$port" \
    --no-browser \
    --ip=127.0.0.1 \
    --notebook-dir="$PROJECT_ROOT/notebooks" \
    > "$LOG_FILE" 2>&1 &

echo $! > "$PID_FILE"

# Wait for startup
while ! curl -s "http://localhost:$port/api" &>/dev/null; do
    sleep 0.5
    if ! kill -0 "$(cat "$PID_FILE")" 2>/dev/null; then
        echo "Server died during startup"
        exit 1
    fi
done

7. Stop Protocol

pid=$(cat "$PID_FILE")

# 1. Send SIGTERM (graceful)
kill -TERM "$pid"

# 2. Wait up to 10 seconds for graceful shutdown
for i in $(seq 1 20); do
    kill -0 "$pid" 2>/dev/null || break
    sleep 0.5
done

# 3. If still alive, SIGKILL
if kill -0 "$pid" 2>/dev/null; then
    kill -KILL "$pid"
fi

# 4. Clean up
rm -f "$PID_FILE"

8. Orphan Detection

The status command SHOULD detect orphan Jupyter processes belonging to this project but not tracked by the PID file:

ps aux | grep "[j]upyter.*--notebook-dir.*$PROJECT_ROOT" | awk '{print $2}'

9. Stale Runtime Cleanup

On every start and status, clean up stale Jupyter runtime JSON files (where the PID is no longer alive):

for json_file in "$JUPYTER_RUNTIME_DIR"/jpserver-*.json; do
    pid=$(python3 -c "import json; print(json.load(open('$json_file')).get('pid',''))")
    if ! kill -0 "$pid" 2>/dev/null; then
        rm -f "$json_file"
    fi
done

10. .gitignore Entries

# Jupyter isolation directories
.jupyter_config/
.jupyter_data/
.jupyter_runtime/
.ipython/
.cache/

# Process management
.logs/
.run/