ADR-036: RVF Model Training Pipeline & UI Integration

[ruvnet]

ADR-036: RVF Model Training Pipeline & UI Integration

Status: Proposed | Date: 2026-03-02

Context

The system currently operates in signal-derived mode — keypoints are computed from aggregate CSI features (motion power, breathing rate, variance) using deterministic math. This gives whole-body presence and gross motion but cannot track individual limbs.

The infrastructure for model inference exists but is disconnected:

• RVF container format (1,102 lines) — binary format with weights, metadata, LoRA profiles, embeddings, witness trails
• Training crate — AdamW, PCK@0.2/OKS metrics, LR scheduling, early stopping, checkpoint export
• NN inference crate — ONNX Runtime with CPU/GPU, dynamic tensors, thread-safe backend
• Sensing server CLI — --model, --train, --pretrain flags exist but end-to-end path not wired

Decision — 5 Phases

Phase 1: Data Collection & Self-Supervised Pretraining

• CSI Recording API: REST endpoints to start/stop/list/download .csi.jsonl recordings
• Contrastive Pretraining (ADR-024): Self-supervised NT-Xent loss → 128-dim CSI embedding, no labels needed
• Output: .rvf with SEG_EMBED + SEG_VEC segments

Phase 2: Supervised Training Pipeline

• Dataset Integration: MM-Fi (114→56 subcarrier interpolation) + Wi-Pose (30→56 zero-pad) + self-collected
• Training API: REST + WebSocket streaming of loss curves, PCK, OKS, early stopping
• RVF Export: Auto-export best checkpoint as .rvf with manifest + witness

Phase 3: LoRA Environment Adaptation

• Fine-tune LoRA adapter weights (rank 4-16) on environment-specific recordings
• 5-10 minutes of data from new room suffices
• Hot-swap profiles without reloading base model

Phase 4: UI Integration

Component	Description
Model Panel	Model library, inspector, load/unload, LoRA profile selector
Training Panel	Recording controls, live loss/PCK charts, epoch table, early stopping indicator
Live Demo Enhancements	Model selector dropdown, LoRA profile switcher, A/B split view (signal vs model), pose trail
Settings Extensions	Model config (device, threads, auto-load), training config (epochs, LR, patience)
Dark Mode	All new panels match existing dark theme

Phase 5: Inference Pipeline Wiring

• CSI frame → amplitude/phase tensors → ONNX session → Kalman smoothing → WebSocket
• Progressive Loading: Layer A (instant signal-derived) → Layer B (embeddings) → Layer C (full model)
• pose_source: "model_inference" badge switches automatically

Implementation Order

Phase	Effort	Priority
1.1 CSI Recording API	2-3 days	High
1.2 Contrastive Pretraining	3-5 days	High
2.1 Dataset Integration	3-5 days	High
2.2 Training API	2-3 days	High
2.3 RVF Export	1-2 days	Medium
3.1 LoRA Fine-Tuning	3-5 days	Medium
3.2 Profile Switching	1 day	Medium
4.1 Model Panel UI	2-3 days	High
4.2 Training Dashboard UI	3-4 days	High
4.3 Live Demo Enhancements	2-3 days	Medium
5.1 Inference Wiring	3-5 days	High
5.2 Progressive Loading	2-3 days	Medium

Total: ~4-6 weeks (phases parallelizable)

Files

New

• ui/components/ModelPanel.js — Model library & inspector
• ui/components/TrainingPanel.js — Recording + training progress
• ui/services/model.service.js — Model API client
• ui/services/training.service.js — Training API client
• sensing-server/src/recording.rs — CSI recording handlers
• sensing-server/src/training_api.rs — Training API + WS progress
• sensing-server/src/model_manager.rs — Model loading & LoRA

Modified

• sensing-server/src/main.rs — Wire new modules, inference branch
• train/src/trainer.rs — WS progress callback, LoRA mode
• train/src/dataset.rs — MM-Fi + Wi-Pose loaders
• nn/src/onnx.rs — LoRA injection, INT8 quantization
• ui/components/LiveDemoTab.js — Model selector, A/B split
• ui/components/SettingsPanel.js — Model/training config + dark mode
• ui/components/PoseDetectionCanvas.js — Pose trail
• ui/index.html — Training tab
• ui/app.js — Tab initialization
• ui/style.css — New panel styles

References

• ADR-036 full text
• ADR-015: MM-Fi + Wi-Pose datasets
• ADR-016: RuVector training pipeline
• ADR-024: AETHER contrastive embeddings
• ADR-035: Live sensing UI accuracy
• Settings panel still uses light theme — convert to dark mode #92 Settings dark mode
• Branch: feat/rvf-training-ui-improvements

[ruvnet]

Closing — ADR-036 RVF training pipeline & UI integration is implemented. PR #96 merged with: model manager, recording API, training API in the Rust sensing server; ModelPanel, TrainingPanel, and training.service.js in the UI; LoRA fine-tuning support. The training pipeline is end-to-end functional.