Time-Series Collector agent

timeseries-collector — subscribes to MQTT (sensors, detections, Home Assistant state changes, Sinergym) and writes the data to SQLite time-series tables for historical queries and ML training. Writes are batched and auto-pruned by a retention window. No LLM.

Dependencies (installed on first spawn): aiomqtt.

Spawn

@catalog spawn timeseries-collector

It's an on-demand agent (not started at boot). Once running it collects continuously and is a natural companion to the Anomaly Detector.

Commands

@timeseries-collector stats     # received/written counts and per-table row counts
@timeseries-collector prune     # force a prune pass now
@timeseries-collector flush     # flush buffered rows to SQLite now

Configuration

Field Meaning
topics MQTT topic patterns to collect (default: sensors/#, detections, HA state, Sinergym)
batch_interval seconds between SQLite flushes (default 5.0)
batch_size buffered-row hint before a flush (default 200)
retention_days auto-prune data older than N days (default 90)
prune_interval_hours hours between prune passes (default 6.0)

Returns total_received, total_written, buffer_sizes, table_rows, and retention_days.

Recording to Parquet, for the model loop

The collector keeps the dashboard's live window in SQLite. Readings the model loop trains on and watches for drift go to Parquet instead, one file per source and hour under the state directory's data/, written by the recorder library agent (docs/design/data-storage.md):

@catalog spawn recorder {"topics": ["sensors/imu/#"], "source_field": "device"}

Or on a node, where it records what that node sees and the node ships each closed file to main:

{"type": "module", "target": "wactorz.agents.recorder_agent:RecorderAgent",
 "name": "recorder", "node": "pi-kitchen",
 "options": {"topics": ["sensors/imu/#"], "source_field": "device"}}

Each reading is a row of its source, the payload's source or device field, or the topic without its wildcards when it names none, filed under its event time with the arrival beside it. wactorz-data sources, wactorz-data profile <source> and GET /api/data/{source}/profile say what was recorded; wactorz-data export writes a window as one file for a notebook. The recorder needs the data extra (pyarrow).

What is recorded is what the trainer library agent trains on: spawned with a recipe, a model's name and a source, it fits a new version on a window of the readings when sent a task, and sets it as the model's shadow when it beats the version served (docs/library.md, "Training a version"; docs/design/train-step.md).

And what the drift library agent watches: spawned with a model, and the source when the served version does not say what it was trained on, it reads the last day and the version's training window through the loader on a cadence, runs PSI and KS on each feature and ADWIN on the model's recorded scores, and raises each shift once, as an observation, a feed line and a message on models/<name>/drift for a rule to retrain on (docs/library.md, "Watching for drift"; docs/design/drift.md):

@catalog spawn drift {"model": "imu-detector", "source": "sensors/imu", "scores": "anomalies/imu"}