The format

The constitution. Everything else composes against this page. schema_version: 1.

An episode is one robot trajectory. On disk it is a bundle:

ep_<32hex>/
├── data.parquet                      # the low-dimensional timeline
├── episode.json                      # sidecar — canonical metadata
├── observation.images.<camera>.mp4   # one video per camera, flat at bundle root
└── …                                 # (one mp4 per additional camera)

A dataset is bundles plus an index:

<dataset>/
├── manifest.json                     # fps, features, episode index
└── episodes/<ep_id>/…                # bundles as above

This layout has exactly two writers — the runtime recorder and koyu pull — and unlimited readers. dataloader.py (vendored in every template) is the reference reader, not the required one.

The vocabulary

Feature keys are the platform's single vocabulary, used identically in parquet columns, video filenames, manifest features dicts, dataloader outputs, policy inputs, and eval messages:

observation.state            low-dim proprioception (float32 vector)
observation.images.<camera>  one camera stream
action                       the action vector

A video's filename is its feature key plus .mp4. No mapping tables exist anywhere, by design.

data.parquet

column	type	meaning
step	int64	0…N−1
timestamp	int64	nanoseconds
observation.state	list\<float32>	one vector per row
action	list\<float32>	one vector per row
(other column features)	list\<float32>	additive

The invariant

Parquet row i corresponds to frame i of every video in the bundle.

Capture is clock-gated: one video frame per row, always. Consequently readers decode by frame index, never by timestamp — no seek math, no fps arithmetic, exact frames. (Encoded fps is the measured capture rate and may be non-integer; because of this invariant, that never matters to a reader.)

episode.json (sidecar)

Canonical per-episode metadata; the bundle directory name is cosmetic. Fields: schema_version, capture_id, recorded_at, length, fps, record_hz, task, task_description, collection_mode (teleop | eval | intervention | synthetic), features (dtype/shape per key), encoding, and provenance: source_run_id, source_checkpoint, policy_name, plus reward (0..1) for episodes that carry a verdict (eval rollouts, rated demos).

manifest.json

{
  "id": "mf_…", "name": "libero-spatial", "type": "teleop", "fps": 20.0,
  "features": { "<key>": {"dtype": "…", "shape": […]}, … },
  "episode_count": 500,
  "success_rate": null,
  "episodes": [ {"id": "ep_…", "length": 97, "task": "…", "task_description": "…"}, … ]
}

length per episode means a reader can build its whole sampling index from this file alone — no bundle I/O at startup.

Encoding (write-time)

libx264, yuv420p, CRF ≈ 20, short GOP. Bundles are committed by atomic directory rename — a torn bundle is never visible. Episodes are immutable once committed; later annotations (dense progress labels, language spans) attach as overlay files, never by rewriting a bundle.

Evolution rules

1. New needs are met by adding — columns, feature keys, sidecar fields — never by changing the meaning of existing ones.
2. Anything that would break a reader bumps schema_version, and readers must refuse versions they do not understand, loudly.
3. The executable twin of this page is the dataloader golden test shipped in every template. If it is green, your data conforms.