Plesty Documentation

Runs, Data & Resume

What an experiment run persists — plan, journal, per-step results — and how crash-safe resume works.

Every run() creates its own checkpoint directory under run_root:

<run_root>/<run_id>/
├── plan.json        # frozen schedule + config + content hash
├── journal.jsonl    # append-only event log (status, resume source)
└── data/            # per-step results: raw blob + JSON metadata
    ├── step_0000.json
    └── step_0000.npy

The run id is derived from the experiment name and a timestamp, e.g. demoexperiment_20260708-101500.

The journal

The journal is a JSON-Lines file — one event per line — recording the life of the run: run_started, step_started, step_completed, step_failed, run_canceled, run_aborted, run_completed. Every append is flushed and fsynced, so the journal survives crashes; at worst the final line is torn, which replay tolerates by skipping it.

There is no mutable "status" file: the current run status is always derived by replaying the journal — exactly like reconstructing training progress from a deep-learning run log.

Per-step data

Each step result is saved as a JSON metadata document plus a raw blob next to it:

  • a PlestyArray → step_NNNN.json (name, unit, description, provenance) + step_NNNN.npy (the array)
  • encoded bytes (e.g. a camera image) → JSON document + binary blob
  • any JSON-serializable value → JSON document only

The JSON document records provenance verbatim — step id, operation, parameters — so every data file is traceable to the exact plan step that produced it.

Resume

uv run python -m plesty.scan_exp --resume demoexperiment_20260708-101500

On resume, the base class:

  1. Rebuilds the plan with build_plan() and compares its content hash against the persisted plan.json — a mismatch (changed config or schedule) is refused with PlanMismatchError
  2. Replays the journal to find the completed step ids
  3. Skips completed steps and continues from the first unfinished one, appending to the same journal

Retries

A failed step is retried up to max_retries times (constructor argument, default 3), sleeping retry_sleep seconds between attempts. Only when a step exhausts its retries does the run abort — journaled as run_aborted, ready to be resumed after the cause is fixed.