Skip to content

Measuring satellite (recorder/dispatcher) intake performance

The recorder and dispatcher ingest transactions from the main opendaf process over transaction-serdes on TCP loopback :6378. This guide covers measuring the 2026-10 intake optimizations on a deployment, and reproducing the loopback smoke locally.

What changed, what to expect

Change Where Expected effect
Geometric MOPS growth + snapshot pre-sizing main, serialization time-to-snapshot after satellite (re)connect drops from O(N²) memcpy to linear; large plants no longer stall the main reactor per reconnect
Serialize-once fan-out main, per commit serialization CPU halves with two satellites
One chunk per transaction + sendv + edge-armed WRITE_MASK + TCP_NODELAY (opt-in: --tcp-nodelay=1 on the serdes links only) main, socket writer send syscalls per transaction: ≥2 → 1; epoll_ctl per message → per burst; tens of ms Nagle latency gone
Periodic deserialization watchdog satellites two timer-heap ops per received transaction → none (5 s tick; timeout detection granularity 10–15 s)
Recycled 64 KiB receive block satellites one alloc/free pair per readable event → none
Unknown-object tolerance satellites CSV drift no longer causes a 5 s reconnect + full-snapshot storm; rate-limited warnings name the object
Dispatcher --request AUTO dispatcher without process-variable rule templates: the measurement/command streams are not requested at all — the dominant dispatcher load disappears on both ends
Archive first-touch filtering, in-place deadband, RTTI-free sampling recorder per-archived-commit allocations drop; string/none objects stop at the first stage

On a deployment

Baseline before upgrading, repeat after, same durations:

PID=$(pidof dafmand || pidof opendaf)       # main process
# syscall profile of the serdes link (sends, epoll_ctl):
strace -f -c -p $PID & sleep 60; kill %1
# CPU of all three processes:
pidstat -p $(pgrep -d, -f 'opendaf|dafmand') 10 6

Time-to-snapshot: restart one satellite and measure from its "Connector(...): Connected" log line to the recorder's "Archive: N initial samples taken" (journalctl timestamps).

Dispatcher advice: after start it logs Dispatcher: Intake advice: alarm state changes only or ... + measurement/command streams (rule templates read process variables). Verify it matches the site's rules; --request AUTO falls back to the full legacy mask until the dispatcher publishes advice.

Loopback smoke (from the build tree, no install)

  1. Scratch profile: a cfgroot with measurements.csv (a few hundred rows, any fake connector name), empty commands.csv/alarms.csv/ alarm-groups.csv, and three service confs modeled on src/lib/dafman/share/svc-recorder-unix.conf / etc/opendaf/default/svc-dispatcher.conf:
  2. main: Core + csv loaders + opendaf-stream-socket-acceptor on 127.0.0.1:16378 with a trnserdes.conf gestalt + CivetWeb + opendaf-rest-api-measurements;
  3. recorder/dispatcher: Core + csv loaders + SourceStream (socket-stream-connection + transaction-serdes --des) + connector to 127.0.0.1:16378 + Archive (scratch LMDB dir) or Dispatcher (empty rules dir, --request AUTO).
  4. export LD_LIBRARY_PATH=$(find build/src -name "libopendaf-*.so" -printf '%h\n' | sort -u | paste -sd:)
  5. Run build/src/opendaf/opendaf -f <conf> for each process; the REST API injects live commits: curl -X POST "http://127.0.0.1:18090/measurements/M_SIM_1?value=f1.5".
  6. Pass criteria: recorder logs the full snapshot ("N initial samples taken") and its data.mdb grows with injected updates; the dispatcher logs its intake advice; no ERROR/WARNING in any log; strace -c on the main process shows roughly one sendto/write per injected commit per full-mask satellite.