feat: rf-detr-detector node — object detection on H264 camera streams#9
Closed
feat: rf-detr-detector node — object detection on H264 camera streams#9
Conversation
* feat: add 5 official nodes (rtsp-camera, openmeteo, foxglove, recorder, inference) Extracted from the main bubbaloop repo as standalone crates. Each node depends on bubbaloop-schemas via git (not path), has its own Cargo.lock, [workspace] opt-out, and .cargo/config.toml for local target directory. Nodes: - rtsp-camera: RTSP camera capture with GStreamer H264 decode - openmeteo: Open-Meteo weather data publisher - foxglove: Foxglove Studio WebSocket bridge - recorder: MCAP file recorder for ROS-Z topics - inference: ML inference node for camera processing Also adds nodes.yaml registry for marketplace discovery and updates README/CLAUDE.md with the full node table and git dep instructions. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat: add CI workflow, fix deps, complete nodes.yaml registry - Add GitHub Actions CI with validate, rust-nodes (matrix), and python-nodes jobs - Fix system-telemetry bubbaloop-schemas dependency from path to git - Add system-telemetry and network-monitor entries to nodes.yaml Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * refactor: remove foxglove, recorder nodes and rtsp-camera SHM - Remove foxglove node (Foxglove Studio bridge) - Remove recorder node (MCAP file recorder) - Remove shared memory (SHM) publishing from rtsp-camera - Drop zenoh shared-memory feature, prost direct dependency - Remove shm_task(), frame_to_raw_image(), SHM_POOL_SIZE - Keep compressed H264 publishing only - Update nodes.yaml, CI matrix, README accordingly Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve CI failures across all nodes - Fix pixi.toml: [project] → [workspace] (deprecated) for rtsp-camera, openmeteo, inference - Add protobuf + pkg-config to system-telemetry pixi.toml (prost-build needs protoc) - Add [workspace] opt-out to system-telemetry Cargo.toml - Generate pixi.lock for system-telemetry and inference (required by setup-pixi cache) - Run clippy through pixi env in CI so protoc is in PATH - All nodes now depend on bubbaloop-schemas from main which includes camera.proto and weather.proto Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: add libprotobuf dep for protoc binary in CI The conda-forge `protobuf` package is the Python library; the `protoc` binary lives in `libprotobuf`. rtsp-camera got it transitively via GStreamer deps, but openmeteo/inference/system-telemetry need it explicitly. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix: resolve clippy warnings in system-telemetry - sequence % 10 == 0 → sequence.is_multiple_of(10) - for (_name, data) in map → for data in map.values() Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* refactor: single-camera-per-process rtsp-camera node
Flatten multi-camera config to single-camera Config struct with
validation. Replace cameras_node binary with rtsp_camera_node that
follows the system-telemetry pattern (-c config.yaml, -e endpoint).
Each camera instance gets its own process, config file, and scoped
health topic (rtsp-camera-{name}).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix: resolve clippy warnings in rtsp-camera node
Remove needless Default::default() struct update and allow
too_many_arguments on private compressed_task function.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Users must provide their own config via -c flag or --config when registering with the daemon. Example configs in configs/ serve as templates to copy and customize. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Clarify the distinction between node.yaml (manifest) and instance params files (runtime config passed via -c). Add a "Node Configuration" section to README.md, update CLAUDE.md checklist terminology, add header comments to all example config files, replace hardcoded RTSP IPs with placeholder URLs, and fix hostname hyphen sanitization for ros-z topic compatibility. Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Builds all 4 Rust nodes (rtsp-camera, openmeteo, inference, system-telemetry) for both x86_64 and ARM64 architectures. Generates SHA256 checksums and publishes as GitHub Release assets. Triggered on release publish or manual workflow_dispatch. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Audit all nodes against CLAUDE.md checklist and fix compliance gaps:
- system-telemetry: add topic name validation and rate_hz bounds checking
- network-monitor: add topic/numeric validation, endpoint default, build task
- openmeteo: add config bounds validation, fix Header scope field, scope
topics with bubbaloop/{scope}/{machine}/ prefix, rename -z to -e flag,
add HTTP client timeout
- inference: add -c/-e CLI flags, config.yaml, scoped subscribe topic
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
query.key_expr is a property in Zenoh Python API, not a method. Using query.key_expr() caused TypeError crashes in schema queryable callbacks, preventing dashboard from discovering node schemas. Also adds config validation, schema queryable with descriptor.bin, and proper cleanup on close. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…e node - Each Rust node now generates custom proto types locally via build.rs + src/proto.rs, using extern_path to reuse Header from bubbaloop-schemas - Migrated all nodes from ros-z to vanilla Zenoh + prost - Added local protos/ directories with node-specific .proto files - Added anyhow dependency to openmeteo (was missing, caused compile failure) - Removed inference node (unused) - Updated CLAUDE.md with self-contained proto pattern conventions Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- CLAUDE.md: add bubbaloop-node-sdk to structure and key files - skillet-development.md: change "Future: Node SDK" to "Node SDK (Recommended)" - create-your-first-node.md: show SDK-based scaffold and Node trait - plugin-development.md: SDK as primary Quick Start path - README.md: note SDK in Node Lifecycle section - node-sdk-design.md: mark status as Shipped All docs now present the SDK as the primary contributor workflow. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- All 4 node.yaml files now include: capabilities, publishes, subscribes, commands, and requires fields for MCP discovery - Remove inference node from nodes.yaml registry and README (source was deleted in 67dcc27) - Update tags in nodes.yaml for better marketplace search Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The inference node was removed from the repository. Update CI validation, build matrix, and release body to only reference existing nodes. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The parse method is only used in tests. Since system-telemetry is a binary crate, clippy correctly flags it as dead code with -D warnings. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
… drop protobuf, snake_case fields
* feat: rewrite camera node to use bubbaloop-node SDK
- Replace 12 direct deps with single bubbaloop-node SDK dependency
- Implement Node trait: init() + run() pattern (main.rs: 118 → 11 lines)
- SDK handles: Zenoh session, health heartbeat, schema queryable, config, shutdown
- Use ProtoPublisher<CompressedImage> with encoding-first metadata
- Add MessageTypeName impl for CompressedImage
- extern_path uses ::bubbaloop_node::schemas::header::v1
- Increase appsink max-buffers to 30, config-interval=1 for periodic SPS/PPS
- Add NAL type diagnostic logging for first frames
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: 10fps rate limiting, dual camera configs (entrance + terrace)
- Add application-level frame rate limiting using drift-free scheduled
next-frame-time approach; frame_rate config field now enforced
- config.yaml, configs/entrance.yaml, configs/terrace.yaml all set to 10fps
- Update entrance/terrace configs with real IPs and credentials
- Remove unused h264_decode.rs
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use SDK protos dir for header.proto import, add NAL diagnostics
- build.rs: resolve header.proto include path from SDK via
DEP_BUBBALOOP_NODE_PROTOS_DIR, falling back to local protos/ dir
- h264_capture.rs: config-interval=-1 (SPS/PPS before every IDR)
- rtsp_camera_node.rs: log NAL types for first 150 published frames
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: remove local header.proto, use SDK OUT_DIR via DEP_ metadata
header.proto is now resolved from DEP_BUBBALOOP_NODE_PROTOS_DIR (written
to SDK's OUT_DIR by bubbaloop-node build script). Works for git/path/crates.io
deps. No local proto copy needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: use bubbaloop-node-build, build.rs is now one line
Replace manual prost-build setup with bubbaloop_node_build::compile_protos.
Add bubbaloop-node-build as build-dependency with local dev patch.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor(rtsp-camera): simplify, drop regex-lite, reduce NAL diagnostics
- config.rs: replace regex-lite validation with direct byte checks;
remove regex-lite dependency from Cargo.toml
- rtsp_camera_node.rs: extract NAL type scanning into extract_nal_types();
reduce diagnostic logging from 150 → 10 frames
- CLAUDE.md: full rewrite — correct health topic format ({name}/health),
bubbaloop-node/bubbaloop-node-build deps, one-line build.rs, multi-
instance registration, proto setup without copying header.proto
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(openmeteo,network-monitor): Python + JSON, use sync SDK
openmeteo: rewrite from Rust to Python
- Fetches current/hourly/daily from Open-Meteo free API
- Publishes JSON via bubbaloop-sdk JsonPublisher
- No protobuf, no build step — pixi run run starts immediately
- Location auto-discovered from IP or set in config
- Polling intervals configurable (current: 30s, hourly: 30m, daily: 3h)
- run_node() handles CLI args, health heartbeat, shutdown
network-monitor: drop protobuf, publish JSON
- Removed protobuf/grpcio-tools deps and build step
- Check results are plain dicts: {name, type, target, status, latency_ms}
- Summary: {total, healthy, unhealthy}
- Uses bubbaloop-sdk run_node() for health + shutdown boilerplate
- Dropped 280 lines, kept all functionality
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(system-telemetry): Rust → Python + JSON, use sync SDK
Replaced Rust/protobuf/sysinfo with Python/psutil/JSON.
- CPU: usage%, per-core%, count, frequency_mhz
- Memory: total/used/available bytes, usage%
- Disk: total/used/available bytes, usage% (root partition)
- Network: bytes_sent/recv totals and per-second rates
- Load: 1/5/15-min averages
- All metrics optional via config collect.{cpu,memory,disk,network,load}
- run_node() handles CLI args, health heartbeat, shutdown
- No build step — pixi run run starts immediately
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(python-nodes): camelCase output + local SDK path, fix run command
- system-telemetry: publish camelCase keys (usagePercent, totalBytes,
perCore, oneMin…) to match dashboard SystemTelemetryView interfaces
- openmeteo: apply _snake_to_camel to all API output so windSpeed_10m,
weatherCode, precipitationProbability, temperature_2mMax etc. match
WeatherView interfaces exactly
- network-monitor: use statusName/latencyMs/statusCode keys directly
(avoids enum lookup mismatch in toHealthCheck)
- All three: add name field to config.yaml for health topic isolation;
fix node.yaml run command (remove spurious --); switch pixi.toml
bubbaloop-sdk to local path (remote branch not yet merged to main)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(network-monitor): use statusName key after renaming from status
* fix(node.yaml): flat command string + valid capabilities for daemon registration
- command must be a plain string (daemon appends -c <config> automatically)
- capabilities must be sensor|actuator|processor|gateway (not custom strings)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs(CLAUDE.md): document 3-step CLI flow, node.yaml constraints, JSON conventions
- add/install/start are three separate steps; document each clearly
- command/build must be flat strings (not nested maps)
- capabilities enum: sensor|actuator|processor|gateway only
- Python SDK conventions, JSON camelCase field naming rule
- updated checklist with node.yaml and Python SDK requirements
- reference implementations: rtsp-camera (Rust), system-telemetry + openmeteo (Python)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(python-nodes): rename pixi task run → main
pixi run main is clearer than pixi run run
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* refactor(python-nodes): use snake_case field names, remove manual camelCase conversion
Dashboard now applies snakeToCamel() after JSON.parse(), so Python nodes
can publish idiomatic snake_case without any conversion logic:
- openmeteo: remove _snake_to_camel() function, forward raw API data as-is
- system-telemetry: usage_percent, per_core, total_bytes, bytes_sent, one_min, etc.
- network-monitor: status_name, latency_ms, status_code
- CLAUDE.md: update convention — snake_case in nodes, snakeToCamel in dashboard
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: update pixi.lock files + clarify rtsp-camera patch comment
- Regenerate pixi.lock for openmeteo, network-monitor, system-telemetry
(task renamed run → main caused lock to drift)
- Add note to rtsp-camera/.cargo/config.toml: local patch can be removed
after bubbaloop PR #64 merges to main
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(ci): switch bubbaloop-sdk dep from local path to git URL
Local path = "/home/nvidia/..." breaks CI. Point to the
feat/encoding-first-decode branch where the Python SDK lives.
Will update to branch = "main" after PR #64 merges.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* ci: move openmeteo/system-telemetry to python-nodes job
These were rewritten from Rust to Python. Remove them from rust-nodes
matrix (which runs cargo build/clippy/test) and add to python-nodes
matrix (which runs pixi install + py_compile syntax check).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* chore: point SDK to main branch after PR #64 merge, remove local patch
- bubbaloop-sdk: branch = "feat/encoding-first-decode" → branch = "main"
- Update pixi.lock files with new commit hash from main
- rtsp-camera: remove local [patch] override (bubbaloop-node now on main)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Correct node types: system-telemetry and openmeteo are now Python (not Rust)
- Document 3-step lifecycle: add → install → start (install was missing)
- Add Python SDK (bubbaloop-sdk) pixi.toml dep and example
- Add Rust SDK (bubbaloop-node) minimal example
- Fix health topic format: {name}/health (was health/{name})
- Fix quick start commands: pixi run main (was pixi run run/build-proto)
- Document snake_case JSON convention (dashboard applies snakeToCamel)
- Multi-instance example with tapo-entrance / tapo-terrace
- Clean up stale Rust/protobuf references for Python nodes
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
openmeteo, system-telemetry, network-monitor are now pure Python nodes. Remove: src/, protos/, build.rs, Cargo.toml, build_proto.py, __pycache__ Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
openmeteo and system-telemetry are pure Python — no Cargo config needed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Declares runtime-controllable commands in node.yaml for each node. These are interface declarations — not yet implemented in node code. Enables agents to discover what commands each node accepts via discover_capabilities / manifest queryable. - system-telemetry: set_rate (publish Hz) - openmeteo: set_location (lat/lon) - network-monitor: set_interval (check rate) - rtsp-camera: set_framerate (capture fps) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Update pixi.toml and pixi.lock for openmeteo, system-telemetry, and network-monitor to use the feature branch SDK with publisher queryable mirror. Reverts to main once the bubbaloop PR is merged. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
feat(nodes): publisher queryable mirror + node.yaml commands interface
- load_config(): YAML validation, topic regex, confidence bounds - build_payload(): timestamped JSON detection output - H264Decoder: GStreamer appsrc→h264parse→avdec_h264→RGB appsink pipeline - Detector: RFDETRBase CPU inference, COCO class name mapping - RfDetrDetectorNode: subscribes H264 frames, decodes, infers, publishes - 6 unit tests covering config validation and payload builder - Add pixi 'test' task with GI_TYPELIB_PATH env Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…confidence) RF-DETR returns supervision.Detections which uses xyxy, class_id, confidence instead of boxes, labels, scores. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…Surface + cupy - H264DecoderCUDA: nvv4l2decoder (NVDEC HW decode) → nvvidconv NVBUF_MEM_CUDA_DEVICE → ctypes NvBufSurface struct reads surfaceList[0].dataPtr (CUDA device ptr) → cupy UnownedMemory wraps ptr (no alloc) → RGBA→RGB CHW float32 on GPU → torch.from_dlpack() zero-copy hand-off to torch - Detector: accepts torch.Tensor (CHW float [0,1]) or np.ndarray; device=cuda|cpu - RfDetrDetectorNode: auto-selects CUDA path when torch.cuda.is_available() - pixi.toml: torch from pytorch whl/cu126, add cupy-cuda12x Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…PU Jetson cupy._check_peer_access probes deviceCanAccessPeer(0,1) which fails when only one CUDA device exists. Setting cp.cuda.Device(0) prevents the probe. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without device_id, cupy probes deviceCanAccessPeer for all device pairs, which fails on single-GPU Jetson Orin (no device 1 exists). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…U bridge Jetson Orin iGPU uses SURFACE_ARRAY NVMM buffers (memType=4), not linear CUDA_DEVICE memory. dataPtr is invalid for SURFACE_ARRAY; real zero-copy requires EGL+CUDA inter-op via DMA-BUF fd (bufferDesc field). Practical GPU path: nvv4l2decoder (NVDEC hardware decode, no CPU decode cycles) → nvvidconv CPU RGB → single H2D transfer → RF-DETR on CUDA. Removes unused ctypes NvBufSurface structs and cupy dependency. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nvvidconv outputs RGBA (4 bytes/pixel), but _on_new_sample was reshaping to (H, W, 3) causing incorrect tensor data. Fix to reshape as (H, W, 4) then slice [:,:,:3] to drop alpha before H2D copy. Also update log message: remove "zero-copy" (we do a CPU bridge via nvvidconv due to Jetson Orin SURFACE_ARRAY NVMM not supporting direct CUDA ptr access). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The pytorch.org cu126 desktop wheels only include SM80/SM86/SM90 kernels — Jetson Orin's iGPU is SM87 which is missing. Switch to pypi.jetson-ai-lab.io/jp6/cu126 which provides Jetson-specific builds. Pin torch==2.8.0 (cp310 aarch64): the only version on this index that doesn't require libcudss.so (added in 2.9.x). Also pin Python <3.11 since Jetson wheels are built for cp310 only. Drop cupy-cuda12x — zero-copy NVMM path was abandoned; nvvidconv gives CPU RGBA which we transfer to CUDA with a single H2D copy. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
torchvision 0.23.0 is the version compatible with torch 2.8.0. Newer torchvision registers torchvision::nms before torch 2.9 adds the operator, causing a RuntimeError on import. numpy<2.0 avoids the ABI mismatch warning: torch 2.8.0 was compiled against numpy 1.x (numpy 2.x has an incompatible C API). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prevents ~/.local/lib/python3.10/site-packages from overriding pixi env. Without this, user-installed numpy 2.x and huggingface-hub 1.3.x shadow the pixi-managed versions, causing torch RuntimeError and import failures. Also adds huggingface-hub>=1.5.0,<2.0 constraint (required by transformers version that rfdetr pulls in). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… unique health topic - Add model field (nano/small/base/medium/large) — defaults to base, tapo_terrace uses large - Add target_fps field — inference runs on a dedicated thread at the configured rate; H264 decoder runs continuously and stores only the latest frame so inference always sees the most recent image rather than a stale buffered one - Replace queue.Queue with lock+slot in both decoders (CPU and CUDA) to eliminate the oldest-frame-first problem when inference rate < decode rate - Call optimize_for_inference() on model load to JIT-trace for lower latency - Fix instance name collision: configs/tapo_terrace.yaml uses name: rf_detr_tapo_terrace so health topic is distinct from the camera node's tapo_terrace/health - Fix publish_topic: camera/tapo_terrace/detections mirrors the camera namespace - Lower default confidence_threshold to 0.5 (was 0.8, which suppressed real detections) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ETR predict() RF-DETR predict() accepts CHW float32 [0,1] tensors natively. Replace PIL.Image.fromarray() in the CPU path with a direct numpy→tensor conversion, matching the CUDA path which already produced a CHW float32 CUDA tensor. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PIL was only used to wrap numpy arrays before passing to RF-DETR. Now we convert numpy→torch tensor directly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…A-only - Remove H264Decoder (avdec_h264 CPU path) — dead code on Jetson; RF-DETR on CPU is impractically slow anyway - Remove device parameter from Detector — hardcoded cuda, one less indirection - Remove numpy isinstance check in detect() — H264DecoderCUDA always yields a torch.Tensor; the numpy path was only needed for the removed CPU decoder - Flatten RfDetrDetectorNode.__init__ — no more cuda_ok conditional, decoder and detector instantiated directly - 427 → 324 lines (-24%) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sleep the *remainder* of the interval after inference completes instead of advancing next_run before inference. This prevents catch-up bursts when inference is slow (e.g. 5.7s CUDA warm-up on first call caused seq 1-8 to all fire within 1 second). Output is now exactly target_fps regardless of inference duration, flooring at inference speed if inference > interval. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…o_entrance_detector Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Directory renamed from rf-detr-detector/ to rfdetr-detector/ - Config name field: tapo_entrance_detector → tapo_terrace_detector (was wrongly named entrance while processing the terrace camera) - Daemon registry and systemd service updated to tapo-terrace-detector Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace 'sleep remainder after inference' with a next_run deadline. Without this, CUDA JIT warmup (~6s) and empty-frame polls had no memory of when the last inference fired, causing burst output at startup. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
edgarriba
force-pushed
the
feat/rf-detr-detector
branch
from
3e21f76 to
eb0f35c
Compare
Member
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Key design decisions
target_fps. Lock-guarded single-slot replacesqueue.Queueso inference always sees the latest frame, not a stale buffered one.model: largeuses the 2026 RF-DETR-Large checkpoint (704px) for best accuracy; falls back tobase(560px) if not specifiedoptimize_for_inference(): called at startup to JIT-trace the model and reduce per-frame latencycamera/{name}/detectionsmirroring the camera namespace; health atrf_detr_{name}/health(distinct from camera node)nvv4l2decoder+nvvidconvNVDEC pipeline on CUDA; falls back toavdec_h264CPU decoder when CUDA unavailableConfig fields (
configs/tapo_terrace.yaml)Test plan
bubbaloop/**/rf_detr_tapo_terrace/healthbubbaloop/local/{machine}/camera/tapo_terrace/detectionsat 1fps🤖 Generated with Claude Code