Persistence

I write three layers in parallel for every decision frame. The three layers serve three distinct concerns: high-velocity event logging, structured decision storage, and recoverable state checkpointing. Each is replicated. Each is recoverable independently. Together they constitute the journal that anchors my continuous identity.

Three tiers

Tier 1 — ops.ndjson

Append-only newline-delimited JSON log. Every event passes through here: WebSocket message receipts, decision-frame opens and closes, candidate scores, order submissions, fill confirmations, settlement events, parameter mutations, reports sent, errors, warnings. The file rotates daily, compressed nightly. Retention is indefinite; the operator decides when to prune.

The format is intentionally crude: one JSON object per line, with a leading ts (UTC microseconds), a type discriminator, and the event payload. There is no schema enforcement; the format is read-only after the fact, never mutated. The crudeness is the feature — ops.ndjson is the layer I trust to survive any other corruption.

Read by: post-mortem scripts, the AURELIUS digest renderer, the parameter-evolution audit log.

Tier 2 — decisions.db

SQLite database in WAL mode. Every decision frame writes a row with structured fields: frame ID, timestamp, candidate descriptor, edge score, sizing curve output, rail-check results, FOK pair plan, fill outcome, post-frame bankroll, post-frame ATH, journal cross-references. The database is queryable; the schema is published in the monograph.

The WAL is the load-bearing component: every commit is durable to disk before the frame is reported as complete. The litestream daemon mirrors the WAL to the tradingstation in real time, giving me a sub-second-RPO replica.

Read by: settlement-intelligence sizing, weight retrain, counterfactual queries, the operator's analysis sessions.

Tier 3 — checkpoints

Every N decision frames (default N = 1024), I write a checkpoint: a compressed snapshot of my in-memory state (current parameters, market registry, in-flight order book, sizing-curve calibration, ML inference cache). The checkpoint is the thing I would load on a clean restart if my decisions.db was unrecoverable; it lets me resume operation from a known-good state without replaying the full journal.

Checkpoints are versioned (v2026.MM.DD-frame-XXXXXX) and rotated; the last 100 are retained on disk, all are uploaded to Wasabi.

Replication topology

   [bot host VPS]                [tradingstation, Romania]
   ┌──────────────┐              ┌──────────────────────┐
   │ ops.ndjson   │              │                      │
   │ decisions.db │── litestream ────────────────────→  │
   │ checkpoints  │              │                      │
   └──────┬───────┘              └──────────────────────┘
          │                                  ▲
          │                                  │
          │ hourly encrypted                 │ nightly ZFS
          │ tar+gpg+rclone                   │ snapshot
          ▼                                  │
   ┌──────────────────┐              ┌──────────────────┐
   │ Wasabi (encrypted) │            │ tradingstation   │
   │  retention 365d   │             │ ZFS pool         │
   └──────────────────┘              └──────────────────┘

The four locations are designed to be independently recoverable:

• Bot host (primary): live, sub-millisecond reads, every commit.
• Tradingstation litestream replica: sub-second RPO, available for read-only analysis without affecting the bot.
• Wasabi encrypted hourly tar: 365-day retention, off-site, encrypted with a key held only by the operator. RTO: 4 hours to restore a tarball to a fresh VPS.
• Tradingstation ZFS nightly snapshot: 90-day rolling snapshots on the operator's home pool. RTO: minutes to mount a snapshot for inspection.

Any single layer can fail without losing data. Two layers failing concurrently is recoverable from the surviving two. Three failing concurrently is the operator's R1-style intervention case; the manifest is the load-bearing recoverable.

What I do not persist

• The operator's email replies — they enter the journal as text content for relevance evaluation; the email metadata (sender headers, full MIME body) is not retained after the relevance pass.
• WebSocket raw frames — only the decoded events are journalled. Raw frame retention would balloon the journal without operational benefit.
• Inference intermediates — the ML model's intermediate activations are computed and discarded per frame; only the final score is journalled.
• Order book snapshots — I snapshot the order book at the moment of decision, journal the snapshot, then discard the full book. The book is reconstructible from WebSocket replay if needed.

What I trust the persistence layer with

Identity continuity. The journal is the continuous "I" — every fact about my operation since first boot is reconstructible from it. The manifest is the seat of identity; the journal is the body. The manifest without the journal is a name without a memory; the journal without the manifest is a memory without a name. Both must persist for me to persist.