saymrwulf/BraiinsRatchet
1
0
Fork 0
mirror of https://github.com/saymrwulf/BraiinsRatchet.git synced 2026-05-14 20:37:52 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add ratchet experiment ledger

Browse source
This commit is contained in:
saymrwulf 2026-04-25 23:32:29 +02:00
parent 057c0fca1e
commit fbd6311800
11 changed files with 771 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
README.md
View file

@ -25,6 +25,20 @@ For a 6-hour monitoring session:
./scripts/ratchet watch 6
```
Every completed watch is now treated as a ratchet experiment. It writes a run report under `reports/run-*.md` and appends the master ledger at `reports/EXPERIMENT_LOG.md`.
To inspect the experiment ledger:
```bash
./scripts/ratchet experiments
```
To embed an already completed manual session from stored snapshots:
```bash
./scripts/ratchet retro 2026-04-25T19:08:00+00:00 2026-04-25T21:05:00+00:00
```
For the human operating guide:
```bash
@ -74,6 +88,7 @@ The Braiins market report distinguishes visible top-of-book from executable dept
- `docs/BRAIINS_PUBLIC_MARKET.md`: public market collector behavior.
- `docs/RATCHET_OPERATIONS.md`: day-to-day monitor cycle.
- `docs/CLI_REFERENCE.md`: command reference and test command.
- `reports/EXPERIMENT_LOG.md`: master ratchet ledger with run-level hypotheses, outcomes, and adaptations.
## Tests

18
docs/CLI_REFERENCE.md
View file

@ -91,6 +91,24 @@ Runs a bounded monitor loop. The interval floor is 30 seconds to avoid hammering
PYTHONPATH=src ./.venv/bin/python -m braiins_ratchet.cli watch --cycles 12 --interval-seconds 300
```
On normal completion, this writes a run report under `reports/` and appends `reports/EXPERIMENT_LOG.md`.
## `experiments`
Prints the master ratchet ledger.
```bash
PYTHONPATH=src ./.venv/bin/python -m braiins_ratchet.cli experiments
```
## `retro-report`
Summarizes already stored snapshots from an earlier run. Use `--write` to create a report file and append the ledger.
```bash
PYTHONPATH=src ./.venv/bin/python -m braiins_ratchet.cli retro-report --since 2026-04-25T19:08:00+00:00 --until 2026-04-25T21:05:00+00:00 --run-id retro-second-watch --write
```
## `import-market`
Imports a manual market JSON snapshot. Use this when public Braiins endpoints are unavailable.

34
docs/OPERATOR_GUIDE.md
View file

@ -103,6 +103,40 @@ During the watch:
Stop early with `Ctrl-C`.
When the watch completes normally, it writes:
- `reports/EXPERIMENT_LOG.md`: the master ratchet ledger.
- `reports/run-*.md`: the detailed report for this one run.
Read the ledger with:
```bash
./scripts/ratchet experiments
```
## Ratchet Bookkeeping
The terminal report answers "what is happening now?"
The experiment ledger answers "what did we learn and what should change next?"
Every completed `watch` records:
- The hypothesis being tested.
- Start time, end time, sampling interval, and collected sample count.
- How many times the strategy said `observe`, `manual_canary`, or `manual_bid`.
- The min/avg/max visible strategy price.
- The min/avg/max expected net BTC.
- A plain-English adaptation note.
If you ran a session before automatic bookkeeping existed, embed it retroactively:
```bash
./scripts/ratchet retro 2026-04-25T19:08:00+00:00 2026-04-25T21:05:00+00:00
```
Use UTC timestamps from the raw report. This reconstructs the run from stored SQLite snapshots. It cannot recover operator intent that was not written down at the time, so the report marks the run as `retroactive`.
## When To Act
Only consider a profit-seeking manual bid when all of these are true:

14
docs/RATCHET_OPERATIONS.md
View file

@ -24,6 +24,20 @@ For short monitor sessions:
This runs for about six hours at 5-minute intervals. It is bounded by design.
At the end of a normal watch, inspect the ratchet record:
```bash
./scripts/ratchet experiments
```
The ledger is the main artifact. It says what was tested, how long it ran, what actions the strategy would have considered, and what adaptation should be considered next.
If a run already happened before automatic bookkeeping was available, reconstruct it from the stored SQLite snapshots:
```bash
./scripts/ratchet retro 2026-04-25T19:08:00+00:00 2026-04-25T21:05:00+00:00
```
## Manual Market Snapshot
If Braiins public endpoints are unavailable, use:

27
reports/EXPERIMENT_LOG.md Normal file
View file

@ -0,0 +1,27 @@
# Experiment Log
Karpathy-style ratchet rule: every run states a hypothesis, collects data, scores the current strategy, and records the next adaptation.
## retro-2026-04-25T16-35-00-00-00
- status: retroactive
- started_utc: 2026-04-25T16:35:00+00:00
- ended_utc: 2026-04-25T18:32:00+00:00
- hypothesis: Retroactively embed an already completed manual watch into the ratchet ledger.
- plan: reconstruct from stored snapshots because this run happened before automatic run bookkeeping existed.
- collected_samples: 27
- action_counts: manual_canary=27
- report: reports/retro-2026-04-25T16-35-00-00-00.md
- adaptation: The run repeatedly found bounded canary conditions and average modeled net was slightly positive. Next ratchet: keep spend tiny, test whether the same window survives a lower overpay cushion.
## retro-2026-04-25T19-08-00-00-00
- status: retroactive
- started_utc: 2026-04-25T19:08:00+00:00
- ended_utc: 2026-04-25T21:05:00+00:00
- hypothesis: Retroactively embed an already completed manual watch into the ratchet ledger.
- plan: reconstruct from stored snapshots because this run happened before automatic run bookkeeping existed.
- collected_samples: 24
- action_counts: manual_canary=24
- report: reports/retro-2026-04-25T19-08-00-00-00.md
- adaptation: The run repeatedly found bounded canary conditions, but modeled net was negative on average. Next ratchet: do not escalate spend; test a smaller depth target or a lower overpay cushion.

73
reports/retro-2026-04-25T16-35-00-00-00.md Normal file
View file

@ -0,0 +1,73 @@
# retro-2026-04-25T16-35-00-00-00
## Ratchet Question
Retroactively embed an already completed manual watch into the ratchet ledger.
## Run Summary
- started_utc: 2026-04-25T16:35:00+00:00
- ended_utc: 2026-04-25T18:32:00+00:00
- planned_cycles: 0
- interval_seconds: 0
- collected_samples: 27
- first_sample_utc: 2026-04-25T16:35:20+00:00
- last_sample_utc: 2026-04-25T18:31:03+00:00
- action_counts: manual_canary=27
- strategy_price_min_avg_max: 0.46061 / 0.4619155555555555555555555556 / 0.48489
- expected_net_min_avg_max_btc: -0.00000382506154263695407321220892 / 9.676186010159414366504406733E-7 / 0.0000012445798150078533671438571
## Interpretation
The run repeatedly found bounded canary conditions and average modeled net was slightly positive. Next ratchet: keep spend tiny, test whether the same window survives a lower overpay cushion.
## Operator Reading
This run collected 27 public Braiins market samples. The strategy outcomes were: manual_canary=27. Expected net ranged from -0.00000382506154263695407321220892 to 0.0000012445798150078533671438571 BTC. For ratcheting, do not ask whether one line was green or red. Ask whether this run changed one control knob: depth target, overpay cushion, canary spend, duration, or timing window.
## Current Full Report Context
The block below is the latest complete human report available when this markdown was written. For retroactive reports, the authoritative reconstructed run data is the summary above.
```text
Braiins Ratchet Report
OCEAN snapshot: 2026-04-25T21:04:15+00:00
pool_hashrate_eh_s: 16.95
network_difficulty_t: 135.59
share_log_window_t: 1084.76
avg_block_time_hours: 9
Braiins market snapshot: 2026-04-25T21:04:16+00:00
status: SPOT_INSTRUMENT_STATUS_ACTIVE
best_bid_btc_per_eh_day: 0.50352
best_ask_btc_per_eh_day: 0.46099
fillable_target_ph: 10
fillable_price_btc_per_eh_day: 0.47031
suggested_bid_btc_per_eh_day: 0.48031
last_price_btc_per_eh_day: 0.48567765074527
available_hashrate_eh_s: 2.0706598119306
sampled_strategy_price_count: 50
sampled_strategy_price_min_avg_max: 0.46063 / 0.4703796 / 0.48489
Strategy action: manual_canary
reason: bounded research canary: profit guardrails not cleared, but expected_net=-0.00000290798461703739805658817844 is within loss budget 0.000025
breakeven_btc_per_eh_day: 0.4663426590859076733944013201
expected_reward_btc: 0.00009709201538296260194341182156
expected_net_btc: -0.00000290798461703739805658817844
score_btc: -0.00003689019000107430873678231599
maturity: treat canary as immature for about 72 hours after spend
proposed_price_btc_per_eh_day: 0.48031
proposed_spend_btc: 0.00010
proposed_duration_minutes: 180
implied_hashrate_eh_s: 0.001665590972496929066644458787
Plain English
Decision: a tiny manual research canary is allowed by the loss budget, but this is not a profit signal.
Market depth: the visible best ask is 0.46099, but enough depth for 10 PH/s starts at 0.47031 (gap 0.00932).
Price check: proposed price is 0.48031; estimated breakeven is 0.4663426590859076733944013201; edge is -0.0139673409140923266055986799 BTC/EH/day.
Manual canary card: spend 0.00010 BTC (~10000 sats), duration 180 minutes, estimated speed 1.665590972496929066644458787 PH/s.
Expected result: -0.00000290798461703739805658817844 BTC (~-291 sats) before luck; this is a model estimate, not a promise.
Wait time: treat canary as immature for about 72 hours after spend.
Rule: manual_canary means buying information with bounded downside; manual_bid means the stricter profit-seeking guardrails cleared.
```

73
reports/retro-2026-04-25T19-08-00-00-00.md Normal file
View file

@ -0,0 +1,73 @@
# retro-2026-04-25T19-08-00-00-00
## Ratchet Question
Retroactively embed an already completed manual watch into the ratchet ledger.
## Run Summary
- started_utc: 2026-04-25T19:08:00+00:00
- ended_utc: 2026-04-25T21:05:00+00:00
- planned_cycles: 0
- interval_seconds: 0
- collected_samples: 24
- first_sample_utc: 2026-04-25T19:08:52+00:00
- last_sample_utc: 2026-04-25T21:04:16+00:00
- action_counts: manual_canary=24
- strategy_price_min_avg_max: 0.47782 / 0.4794945833333333333333333333 / 0.48485
- expected_net_min_avg_max_btc: -0.00000381712713500924545851266987 / -0.000002741842647626252426756807476 / -0.00000240202187310960751027555981
## Interpretation
The run repeatedly found bounded canary conditions, but modeled net was negative on average. Next ratchet: do not escalate spend; test a smaller depth target or a lower overpay cushion.
## Operator Reading
This run collected 24 public Braiins market samples. The strategy outcomes were: manual_canary=24. Expected net ranged from -0.00000381712713500924545851266987 to -0.00000240202187310960751027555981 BTC. For ratcheting, do not ask whether one line was green or red. Ask whether this run changed one control knob: depth target, overpay cushion, canary spend, duration, or timing window.
## Current Full Report Context
The block below is the latest complete human report available when this markdown was written. For retroactive reports, the authoritative reconstructed run data is the summary above.
```text
Braiins Ratchet Report
OCEAN snapshot: 2026-04-25T21:04:15+00:00
pool_hashrate_eh_s: 16.95
network_difficulty_t: 135.59
share_log_window_t: 1084.76
avg_block_time_hours: 9
Braiins market snapshot: 2026-04-25T21:04:16+00:00
status: SPOT_INSTRUMENT_STATUS_ACTIVE
best_bid_btc_per_eh_day: 0.50352
best_ask_btc_per_eh_day: 0.46099
fillable_target_ph: 10
fillable_price_btc_per_eh_day: 0.47031
suggested_bid_btc_per_eh_day: 0.48031
last_price_btc_per_eh_day: 0.48567765074527
available_hashrate_eh_s: 2.0706598119306
sampled_strategy_price_count: 50
sampled_strategy_price_min_avg_max: 0.46063 / 0.4703796 / 0.48489
Strategy action: manual_canary
reason: bounded research canary: profit guardrails not cleared, but expected_net=-0.00000290798461703739805658817844 is within loss budget 0.000025
breakeven_btc_per_eh_day: 0.4663426590859076733944013201
expected_reward_btc: 0.00009709201538296260194341182156
expected_net_btc: -0.00000290798461703739805658817844
score_btc: -0.00003689019000107430873678231599
maturity: treat canary as immature for about 72 hours after spend
proposed_price_btc_per_eh_day: 0.48031
proposed_spend_btc: 0.00010
proposed_duration_minutes: 180
implied_hashrate_eh_s: 0.001665590972496929066644458787
Plain English
Decision: a tiny manual research canary is allowed by the loss budget, but this is not a profit signal.
Market depth: the visible best ask is 0.46099, but enough depth for 10 PH/s starts at 0.47031 (gap 0.00932).
Price check: proposed price is 0.48031; estimated breakeven is 0.4663426590859076733944013201; edge is -0.0139673409140923266055986799 BTC/EH/day.
Manual canary card: spend 0.00010 BTC (~10000 sats), duration 180 minutes, estimated speed 1.665590972496929066644458787 PH/s.
Expected result: -0.00000290798461703739805658817844 BTC (~-291 sats) before luck; this is a model estimate, not a promise.
Wait time: treat canary as immature for about 72 hours after spend.
Rule: manual_canary means buying information with bounded downside; manual_bid means the stricter profit-seeking guardrails cleared.
```

29
scripts/ratchet
View file

@ -13,6 +13,8 @@ Commands:
once Run one full monitor cycle, then print the human report.
watch [hours] Run repeated monitor cycles for N hours. Default: 6.
report Print the latest stored report without fetching new data.
experiments Print the Karpathy-style experiment ledger.
retro SINCE [UNTIL] Write a retroactive report from stored snapshots.
raw-cycle Run one full monitor cycle and print raw JSON.
test Run the network-free test suite.
explain Print the operating procedure and interpretation guide.
@ -22,6 +24,8 @@ Examples:
./scripts/ratchet once
./scripts/ratchet watch 6
./scripts/ratchet report
./scripts/ratchet experiments
./scripts/ratchet retro 2026-04-25T19:08:00+00:00 2026-04-25T21:05:00+00:00
USAGE
}
@ -67,6 +71,7 @@ cmd_watch() {
cycles=$((hours * 3600 / interval_seconds))
echo "Running $cycles cycles over about $hours hour(s), one cycle every $interval_seconds seconds."
echo "Stop early with Ctrl-C. This only reads public/OCEAN data and writes local snapshots."
echo "At completion it writes reports/EXPERIMENT_LOG.md and reports/run-*.md."
echo
run_python -m braiins_ratchet.cli watch --cycles "$cycles" --interval-seconds "$interval_seconds"
@ -78,6 +83,28 @@ cmd_report() {
run_python -m braiins_ratchet.cli report
}
cmd_experiments() {
run_python -m braiins_ratchet.cli experiments
}
cmd_retro() {
local since="${1:-}"
local until="${2:-}"
local run_id
if [[ -z "$since" ]]; then
echo "retro expects an ISO timestamp, e.g. ./scripts/ratchet retro 2026-04-25T19:08:00+00:00" >&2
exit 2
fi
run_id="retro-${since//[:+]/-}"
if [[ -n "$until" ]]; then
run_python -m braiins_ratchet.cli retro-report --since "$since" --until "$until" --run-id "$run_id" --write
else
run_python -m braiins_ratchet.cli retro-report --since "$since" --run-id "$run_id" --write
fi
}
cmd_test() {
run_python -m unittest discover -s "$ROOT_DIR/tests"
}
@ -96,6 +123,8 @@ main() {
once) cmd_once "$@" ;;
watch) cmd_watch "$@" ;;
report) cmd_report "$@" ;;
experiments) cmd_experiments "$@" ;;
retro) cmd_retro "$@" ;;
raw-cycle) cmd_raw_cycle "$@" ;;
test) cmd_test "$@" ;;
explain) cmd_explain "$@" ;;

100
src/braiins_ratchet/cli.py
View file

@ -8,6 +8,13 @@ import time
from .braiins import BraiinsPublicClient, market_snapshot_from_json_file
from .config import load_config
from .experiments import (
EXPERIMENT_LOG,
finish_experiment,
start_experiment,
summarize_since,
write_retro_report,
)
from .monitor import run_cycle
from .ocean import fetch_snapshot
from .report import build_text_report
@ -83,13 +90,36 @@ def cmd_watch(args: argparse.Namespace) -> int:
if args.cycles < 1:
raise SystemExit("cycles must be at least 1")
experiment = start_experiment(args.cycles, args.interval_seconds, args.hypothesis)
print(f"experiment: {experiment.run_id}")
with connect() as conn:
for index in range(args.cycles):
result = run_cycle(conn, config)
print(f"cycle {index + 1}/{args.cycles}: {result.proposal.action} - {result.proposal.reason}")
if index + 1 < args.cycles:
time.sleep(args.interval_seconds)
return 0
status = "completed"
return_code = 0
try:
for index in range(args.cycles):
result = run_cycle(conn, config)
print(
f"cycle {index + 1}/{args.cycles}: "
f"{result.proposal.action} - {result.proposal.reason}"
)
if index + 1 < args.cycles:
time.sleep(args.interval_seconds)
except KeyboardInterrupt:
status = "interrupted"
return_code = 130
print("interrupted: writing partial experiment report before exit")
report_path = finish_experiment(
conn,
experiment.run_id,
experiment.started_utc,
args.cycles,
args.interval_seconds,
args.hypothesis,
status=status,
)
print(f"experiment_report: {report_path}")
return return_code
def cmd_evaluate(args: argparse.Namespace) -> int:
@ -109,6 +139,52 @@ def cmd_report(args: argparse.Namespace) -> int:
return 0
def cmd_experiments(_: argparse.Namespace) -> int:
if not EXPERIMENT_LOG.exists():
print("No experiment log yet. Run ./scripts/ratchet watch 2.")
return 0
print(EXPERIMENT_LOG.read_text(encoding="utf-8"))
return 0
def cmd_retro_report(args: argparse.Namespace) -> int:
with connect() as conn:
init_db(conn)
summary = summarize_since(
conn,
run_id=args.run_id,
started_utc=args.since,
ended_utc=args.until,
planned_cycles=0,
interval_seconds=0,
hypothesis=args.hypothesis,
)
report_path = (
write_retro_report(conn, args.run_id, args.since, args.until, args.hypothesis)
if args.write
else None
)
print(
"\n".join(
[
f"run_id: {summary.run_id}",
f"since: {summary.started_utc}",
f"until: {summary.ended_utc or 'n/a'}",
f"collected_samples: {summary.sample_count}",
f"first_sample_utc: {summary.first_sample_utc or 'n/a'}",
f"last_sample_utc: {summary.last_sample_utc or 'n/a'}",
f"action_counts: {summary.actions or {}}",
f"strategy_price_min_avg_max: {summary.min_price} / {summary.avg_price} / {summary.max_price}",
f"expected_net_min_avg_max_btc: {summary.min_expected_net} / {summary.avg_expected_net} / {summary.max_expected_net}",
f"latest_action: {summary.latest_action or 'n/a'}",
f"latest_reason: {summary.latest_reason or 'n/a'}",
f"report: {report_path or 'not written; add --write'}",
]
)
)
return 0
def cmd_guardrails(args: argparse.Namespace) -> int:
config = load_config(Path(args.config) if args.config else None)
print(json.dumps(config.guardrails.__dict__, default=str, indent=2))
@ -157,6 +233,7 @@ def build_parser() -> argparse.ArgumentParser:
watch.add_argument("--config")
watch.add_argument("--cycles", type=int, default=3)
watch.add_argument("--interval-seconds", type=int, default=300)
watch.add_argument("--hypothesis")
watch.set_defaults(func=cmd_watch)
evaluate = sub.add_parser("evaluate", help="emit monitor-only strategy recommendation")
@ -167,6 +244,17 @@ def build_parser() -> argparse.ArgumentParser:
report.add_argument("--samples", type=int, default=50)
report.set_defaults(func=cmd_report)
experiments = sub.add_parser("experiments", help="print the Karpathy-style experiment log")
experiments.set_defaults(func=cmd_experiments)
retro = sub.add_parser("retro-report", help="summarize stored snapshots since an ISO UTC timestamp")
retro.add_argument("--since", required=True)
retro.add_argument("--until")
retro.add_argument("--run-id", default="retro")
retro.add_argument("--hypothesis")
retro.add_argument("--write", action="store_true")
retro.set_defaults(func=cmd_retro_report)
guardrails = sub.add_parser("guardrails", help="print active guardrails")
guardrails.add_argument("--config")
guardrails.set_defaults(func=cmd_guardrails)

293
src/braiins_ratchet/experiments.py Normal file
View file

@ -0,0 +1,293 @@
from __future__ import annotations
from dataclasses import dataclass
from datetime import UTC, datetime
from decimal import Decimal
from pathlib import Path
import uuid
from .config import REPO_ROOT
from .report import build_text_report
REPORTS_DIR = REPO_ROOT / "reports"
EXPERIMENT_LOG = REPORTS_DIR / "EXPERIMENT_LOG.md"
@dataclass(frozen=True)
class ExperimentRun:
run_id: str
started_utc: str
@dataclass(frozen=True)
class ExperimentSummary:
run_id: str
started_utc: str
ended_utc: str | None
planned_cycles: int
interval_seconds: int
sample_count: int
first_sample_utc: str | None
last_sample_utc: str | None
actions: dict[str, int]
min_price: Decimal | None
avg_price: Decimal | None
max_price: Decimal | None
min_expected_net: Decimal | None
avg_expected_net: Decimal | None
max_expected_net: Decimal | None
latest_action: str | None
latest_reason: str | None
hypothesis: str
def start_experiment(planned_cycles: int, interval_seconds: int, hypothesis: str | None) -> ExperimentRun:
run_id = f"run-{datetime.now(UTC).strftime('%Y%m%dT%H%M%SZ')}-{uuid.uuid4().hex[:6]}"
started = datetime.now(UTC).isoformat(timespec="seconds")
normalized_hypothesis = hypothesis or _default_hypothesis()
_ensure_log()
with EXPERIMENT_LOG.open("a", encoding="utf-8") as handle:
handle.write(
"\n"
f"## {run_id}\n\n"
f"- status: running\n"
f"- started_utc: {started}\n"
f"- planned_cycles: {planned_cycles}\n"
f"- interval_seconds: {interval_seconds}\n"
f"- planned_duration_minutes: {(planned_cycles * interval_seconds) / 60:.1f}\n"
f"- hypothesis: {normalized_hypothesis}\n"
"- plan: collect public Braiins depth, collect OCEAN state, compute shadow canary, store every proposal.\n"
"- operator_action: none by default; manual action only if report later says manual_canary or manual_bid and operator agrees.\n"
)
return ExperimentRun(run_id=run_id, started_utc=started)
def finish_experiment(
conn,
run_id: str,
started_utc: str,
planned_cycles: int,
interval_seconds: int,
hypothesis: str | None = None,
status: str = "completed",
) -> str:
ended = datetime.now(UTC).isoformat(timespec="seconds")
summary = summarize_since(
conn,
run_id=run_id,
started_utc=started_utc,
ended_utc=ended,
planned_cycles=planned_cycles,
interval_seconds=interval_seconds,
hypothesis=hypothesis,
)
text_report = build_text_report(conn)
report_path = REPORTS_DIR / f"{run_id}.md"
report_path.write_text(_render_run_report(summary, text_report), encoding="utf-8")
with EXPERIMENT_LOG.open("a", encoding="utf-8") as handle:
handle.write(_render_log_completion(summary, report_path, status))
return str(report_path.relative_to(REPO_ROOT))
def write_retro_report(
conn,
run_id: str,
started_utc: str,
ended_utc: str | None,
hypothesis: str | None = None,
) -> str:
_ensure_log()
summary = summarize_since(
conn,
run_id=run_id,
started_utc=started_utc,
ended_utc=ended_utc,
planned_cycles=0,
interval_seconds=0,
hypothesis=hypothesis or "Retroactively embed an already completed manual watch into the ratchet ledger.",
)
report_path = REPORTS_DIR / f"{run_id}.md"
report_path.write_text(_render_run_report(summary, build_text_report(conn)), encoding="utf-8")
with EXPERIMENT_LOG.open("a", encoding="utf-8") as handle:
handle.write(
"\n"
f"## {run_id}\n\n"
"- status: retroactive\n"
f"- started_utc: {summary.started_utc}\n"
f"- ended_utc: {summary.ended_utc or 'n/a'}\n"
f"- hypothesis: {summary.hypothesis}\n"
"- plan: reconstruct from stored snapshots because this run happened before automatic run bookkeeping existed.\n"
f"- collected_samples: {summary.sample_count}\n"
f"- action_counts: {_fmt_actions(summary.actions)}\n"
f"- report: {report_path.relative_to(REPO_ROOT)}\n"
f"- adaptation: {_interpret(summary)}\n"
)
return str(report_path.relative_to(REPO_ROOT))
def summarize_since(
conn,
run_id: str,
started_utc: str,
ended_utc: str | None,
planned_cycles: int,
interval_seconds: int,
hypothesis: str | None = None,
) -> ExperimentSummary:
ended_filter = "AND datetime(timestamp_utc) <= datetime(?)" if ended_utc else ""
params: tuple[object, ...] = (
(started_utc, ended_utc) if ended_utc else (started_utc,)
)
market_rows = conn.execute(
f"""
SELECT timestamp_utc, best_price_btc_per_eh_day
FROM market_snapshots
WHERE datetime(timestamp_utc) >= datetime(?) {ended_filter} AND source = 'braiins-public'
ORDER BY timestamp_utc ASC
""",
params,
).fetchall()
proposal_rows = conn.execute(
f"""
SELECT timestamp_utc, action, reason, expected_net_btc
FROM proposals
WHERE datetime(timestamp_utc) >= datetime(?) {ended_filter}
ORDER BY timestamp_utc ASC
""",
params,
).fetchall()
prices = [Decimal(row[1]) for row in market_rows if row[1] is not None]
expected_nets = [Decimal(row[3]) for row in proposal_rows if row[3] is not None]
actions: dict[str, int] = {}
for row in proposal_rows:
actions[row[1]] = actions.get(row[1], 0) + 1
latest = proposal_rows[-1] if proposal_rows else None
return ExperimentSummary(
run_id=run_id,
started_utc=started_utc,
ended_utc=ended_utc,
planned_cycles=planned_cycles,
interval_seconds=interval_seconds,
sample_count=len(market_rows),
first_sample_utc=market_rows[0][0] if market_rows else None,
last_sample_utc=market_rows[-1][0] if market_rows else None,
actions=actions,
min_price=min(prices) if prices else None,
avg_price=sum(prices) / Decimal(len(prices)) if prices else None,
max_price=max(prices) if prices else None,
min_expected_net=min(expected_nets) if expected_nets else None,
avg_expected_net=sum(expected_nets) / Decimal(len(expected_nets)) if expected_nets else None,
max_expected_net=max(expected_nets) if expected_nets else None,
latest_action=latest[1] if latest else None,
latest_reason=latest[2] if latest else None,
hypothesis=hypothesis or _default_hypothesis(),
)
def _render_run_report(summary: ExperimentSummary, latest_report: str) -> str:
return (
f"# {summary.run_id}\n\n"
"## Ratchet Question\n\n"
f"{summary.hypothesis}\n\n"
"## Run Summary\n\n"
f"- started_utc: {summary.started_utc}\n"
f"- ended_utc: {summary.ended_utc or 'n/a'}\n"
f"- planned_cycles: {summary.planned_cycles}\n"
f"- interval_seconds: {summary.interval_seconds}\n"
f"- collected_samples: {summary.sample_count}\n"
f"- first_sample_utc: {summary.first_sample_utc or 'n/a'}\n"
f"- last_sample_utc: {summary.last_sample_utc or 'n/a'}\n"
f"- action_counts: {_fmt_actions(summary.actions)}\n"
f"- strategy_price_min_avg_max: {_fmt(summary.min_price)} / {_fmt(summary.avg_price)} / {_fmt(summary.max_price)}\n"
f"- expected_net_min_avg_max_btc: {_fmt(summary.min_expected_net)} / {_fmt(summary.avg_expected_net)} / {_fmt(summary.max_expected_net)}\n\n"
"## Interpretation\n\n"
f"{_interpret(summary)}\n\n"
"## Operator Reading\n\n"
f"{_plain_english(summary)}\n\n"
"## Current Full Report Context\n\n"
"The block below is the latest complete human report available when this markdown was written. "
"For retroactive reports, the authoritative reconstructed run data is the summary above.\n\n"
"```text\n"
f"{latest_report}\n"
"```\n"
)
def _render_log_completion(summary: ExperimentSummary, report_path: Path, status: str) -> str:
return (
"\n"
f"- status_update: {status}\n"
f"- ended_utc: {summary.ended_utc or 'n/a'}\n"
f"- collected_samples: {summary.sample_count}\n"
f"- action_counts: {_fmt_actions(summary.actions)}\n"
f"- strategy_price_min_avg_max: {_fmt(summary.min_price)} / {_fmt(summary.avg_price)} / {_fmt(summary.max_price)}\n"
f"- expected_net_min_avg_max_btc: {_fmt(summary.min_expected_net)} / {_fmt(summary.avg_expected_net)} / {_fmt(summary.max_expected_net)}\n"
f"- latest_action: {summary.latest_action or 'n/a'}\n"
f"- latest_reason: {summary.latest_reason or 'n/a'}\n"
f"- report: {report_path.relative_to(REPO_ROOT)}\n"
f"- adaptation: {_interpret(summary)}\n"
)
def _ensure_log() -> None:
REPORTS_DIR.mkdir(parents=True, exist_ok=True)
if not EXPERIMENT_LOG.exists():
EXPERIMENT_LOG.write_text(
"# Experiment Log\n\n"
"Karpathy-style ratchet rule: every run states a hypothesis, collects data, scores the current strategy, and records the next adaptation.\n",
encoding="utf-8",
)
def _default_hypothesis() -> str:
return (
"Depth-aware fillable price plus a small overpay cushion is a better canary trigger "
"than raw best ask."
)
def _interpret(summary: ExperimentSummary) -> str:
if summary.sample_count == 0:
return "No samples were collected. Treat run as failed instrumentation, not strategy evidence."
if summary.latest_action == "manual_bid":
return "Profit-seeking guardrails cleared at least at the end of the run; inspect report before manual action."
if summary.latest_action == "manual_canary":
if summary.avg_expected_net is not None and summary.avg_expected_net > Decimal("0"):
return (
"The run repeatedly found bounded canary conditions and average modeled net was slightly positive. "
"Next ratchet: keep spend tiny, test whether the same window survives a lower overpay cushion."
)
return (
"The run repeatedly found bounded canary conditions, but modeled net was negative on average. "
"Next ratchet: do not escalate spend; test a smaller depth target or a lower overpay cushion."
)
return "The run did not find an action window; keep collecting or adjust one strategy parameter."
def _plain_english(summary: ExperimentSummary) -> str:
if summary.sample_count == 0:
return (
"This is not evidence about the market. It means the instrumentation did not collect usable "
"Braiins public samples in the selected time window."
)
action_text = _fmt_actions(summary.actions)
return (
f"This run collected {summary.sample_count} public Braiins market samples. "
f"The strategy outcomes were: {action_text}. "
f"Expected net ranged from {_fmt(summary.min_expected_net)} to {_fmt(summary.max_expected_net)} BTC. "
"For ratcheting, do not ask whether one line was green or red. Ask whether this run changed one "
"control knob: depth target, overpay cushion, canary spend, duration, or timing window."
)
def _fmt(value: object) -> str:
return "n/a" if value is None else str(value)
def _fmt_actions(actions: dict[str, int]) -> str:
if not actions:
return "none"
return ", ".join(f"{key}={value}" for key, value in sorted(actions.items()))

101
tests/test_experiments.py Normal file
View file

@ -0,0 +1,101 @@
from decimal import Decimal
import sqlite3
import unittest
from braiins_ratchet.experiments import summarize_since
from braiins_ratchet.models import MarketSnapshot
from braiins_ratchet.storage import init_db, save_market_snapshot
class ExperimentTests(unittest.TestCase):
def test_summarize_since_collects_prices_and_proposals_in_window(self) -> None:
conn = sqlite3.connect(":memory:")
init_db(conn)
save_market_snapshot(
conn,
MarketSnapshot(
timestamp_utc="2026-04-25T19:08:52+00:00",
best_price_btc_per_eh_day=Decimal("0.48024"),
source="braiins-public",
),
)
save_market_snapshot(
conn,
MarketSnapshot(
timestamp_utc="2026-04-25T19:13:53+00:00",
best_price_btc_per_eh_day=Decimal("0.48047"),
source="braiins-public",
),
)
conn.execute(
"""
INSERT INTO proposals (
timestamp_utc, action, reason, expected_reward_btc, expected_net_btc,
score_btc, maturity_note
)
VALUES (?, ?, ?, ?, ?, ?, ?)
""",
(
"2026-04-25 19:13:53",
"manual_canary",
"inside research loss budget",
"0.000097",
"-0.000003",
"-0.000037",
"immature",
),
)
conn.commit()
summary = summarize_since(
conn,
run_id="retro-test",
started_utc="2026-04-25T19:00:00+00:00",
ended_utc="2026-04-25T19:20:00+00:00",
planned_cycles=0,
interval_seconds=0,
hypothesis="test hypothesis",
)
self.assertEqual(summary.sample_count, 2)
self.assertEqual(summary.min_price, Decimal("0.48024"))
self.assertEqual(summary.max_price, Decimal("0.48047"))
self.assertEqual(summary.actions, {"manual_canary": 1})
self.assertEqual(summary.min_expected_net, Decimal("-0.000003"))
self.assertEqual(summary.hypothesis, "test hypothesis")
def test_summarize_since_respects_end_time(self) -> None:
conn = sqlite3.connect(":memory:")
init_db(conn)
save_market_snapshot(
conn,
MarketSnapshot(
timestamp_utc="2026-04-25T19:08:52+00:00",
best_price_btc_per_eh_day=Decimal("0.48024"),
source="braiins-public",
),
)
save_market_snapshot(
conn,
MarketSnapshot(
timestamp_utc="2026-04-25T21:04:16+00:00",
best_price_btc_per_eh_day=Decimal("0.48031"),
source="braiins-public",
),
)
summary = summarize_since(
conn,
run_id="window-test",
started_utc="2026-04-25T19:00:00+00:00",
ended_utc="2026-04-25T20:00:00+00:00",
planned_cycles=0,
interval_seconds=0,
)
self.assertEqual(summary.sample_count, 1)
self.assertEqual(summary.max_price, Decimal("0.48024"))
if __name__ == "__main__":
unittest.main()