feat: integrate sproink Rust spreading activation via CGO

[nvandessel]

Summary

• Replaces floop's per-node-per-step SQLite edge queries with a single FFI call into sproink's in-memory CSR graph for spreading activation
• Introduces Activator interface with NativeEngine (sproink via CGO) and existing Engine (pure-Go fallback)
• Auto-detects CGO availability at startup — falls back gracefully on CGO_ENABLED=0 or missing libsproink
• Adds GetAllEdges and Version() to ExtendedGraphStore for bulk graph loading and staleness detection

Architecture

floop_active call
  → LanceDB pre-filter (unchanged)
  → NativeEngine.Activate()
    → version check → auto-rebuild if stale
    → IDMap: UUID → u32
    → sproink_activate() (single FFI call, entire propagation in Rust)
    → IDMap: u32 → UUID
    → seed source attribution
  → Hebbian learning (Go-side, unchanged)

Key Design Decisions

• Temporal decay: pre-applied in Go during graph load (sproink FFI doesn't expose it yet)
• Feature affinity: virtual edges pre-materialized during graph build
• Graph lifecycle: version counter on store, synchronous rebuild on staleness (sub-ms for floop graph sizes)
• Co-activation pairs: Go-side functions retained for now; FFI wrappers implemented and tested for parity but not wired into handler

Files (23 changed, +2381/-11)

Area	Files
Interface	activator.go, engine.go (conformance check)
ID mapping	idmap.go, edgekind.go
CGO bindings	sproink_cgo.go, sproink_nocgo.go, sproink_pairs_cgo.go
Graph loading	graph_loader.go
Store	store.go, sqlite.go, sqlite_extended.go, multi.go
Integration	server.go, handler_active.go
Tests	8 test files, 30+ test functions including parity and race detection

Test plan

• CGO_ENABLED=1 go test ./internal/spreading/... -race — all pass
• CGO_ENABLED=0 go test ./internal/spreading/... — nocgo stub tests pass
• go test ./internal/store/... — store tests pass
• Parity tests: NativeEngine vs Engine produce equivalent results (8 scenarios)
• Property: boundary tolerance for nodes near MinActivation threshold
• CI: needs Rust toolchain for cross-platform builds (tracked in floop-s8l)

Follow-up beads

• floop-s8l (P2) — CI: Add Rust toolchain for sproink builds
• floop-1wi (P3) — Wire FFI pair extraction into handler
• floop-5ba (P3) — Expose temporal decay + step snapshots via sproink FFI
• floop-gkd (P3) — Performance benchmarks

🤖 Generated with Claude Code

[codecov-commenter]

Codecov Report

❌ Patch coverage is 75.00000% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.75%. Comparing base (8bf6c8d) to head (6d6c6d9).

Files with missing lines	Patch %	Lines
internal/store/sqlite.go	69.09%	10 Missing and 7 partials ⚠️
internal/store/multi.go	64.28%	5 Missing and 5 partials ⚠️
internal/mcp/server.go	61.11%	5 Missing and 2 partials ⚠️
internal/store/sqlite_extended.go	66.66%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #268      +/-   ##
==========================================
- Coverage   78.81%   78.75%   -0.06%     
==========================================
  Files         186      189       +3     
  Lines       18646    18779     +133     
==========================================
+ Hits        14695    14789      +94     
- Misses       2731     2757      +26     
- Partials     1220     1233      +13     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Greptile Summary

This PR replaces per-node SQLite edge queries with a single FFI call into a Rust CSR graph (sproink) for spreading activation, introducing a NativeEngine/Activator interface with graceful pure-Go fallback, and adding GetAllEdges/Version() to ExtendedGraphStore. All previously-flagged P0/P1 issues (version data race, TOCTOU in Rebuild and construction, double-free, thundering herd, multi-store parity gap, CI cache logic, silent global-store error degradation) are resolved in the follow-up commits.

Confidence Score: 5/5

• Safe to merge; all prior P0/P1 findings are fixed and only two P2 style suggestions remain.
• Every critical concurrency, memory-safety, and correctness issue raised in previous review rounds has been addressed. The two remaining findings are P2: a defensive bounds check for FFI-returned node IDs (IDMap.ToUUID panics on out-of-range) and a CI cache key that only captures header changes and not Rust source changes. Neither blocks correctness in normal operation.
• internal/spreading/sproink_cgo.go (ToUUID panic on FFI out-of-range), .github/workflows/ci.yml (cache key gap)

Important Files Changed

Filename	Overview
internal/spreading/sproink_cgo.go	Core NativeEngine implementation: all previously flagged issues (version race, TOCTOU, double-free, thundering herd) are fixed; one P2 remains — IDMap.ToUUID panics on out-of-range FFI node IDs without a defensive bounds check.
internal/spreading/graph_loader.go	Bulk graph loading with temporal decay, affinity edge materialisation, and empty-store handling; correct TOCTOU-safe pattern (version snapshot before reading edges) applied by callers.
internal/store/multi.go	GetAllEdges and Version() now merge local + global stores; Version() uses addition (monotonically non-decreasing), global GetAllEdges errors are propagated rather than silently swallowed.
internal/store/sqlite.go	Adds GetAllEdges, Version(), and bumpVersion(); all write paths (AddNode, UpdateNode, DeleteNode, AddEdge, RemoveEdge) correctly bump the atomic version counter.
internal/mcp/server.go	NativeEngine wired into Server; io.Closer Close() called on shutdown after workerWg.Wait() ensuring no active activations race with cleanup.
.github/workflows/ci.yml	All prior CI cache-logic bugs fixed; one P2 remains — cache key based on sproink.h hash does not detect Rust source-only changes, meaning a bug-fix in sproink without a header change reuses the stale cached binary.
internal/spreading/idmap.go	Clean bidirectional UUID ↔ uint32 mapping; ToUUID intentionally panics on out-of-range — defensive callers needed at FFI boundary.
internal/store/sqlite_extended.go	TouchEdges and BatchUpdateEdgeWeights now correctly bump version; PruneWeakEdges also bumped.
internal/spreading/sproink_nocgo.go	Correct stub for CGO_ENABLED=0 builds; NewNativeEngine returns error, activator falls back to pure-Go Engine.
internal/spreading/sproink_pairs_cgo.go	FFI pair extraction and Oja update wrappers; nativeActivateAndExtractPairs correctly holds RLock before accessing idmap/graph; NativeExtractPairs exported but parameter type (C.SproinkResults) is inaccessible from outside the package.
internal/spreading/parity_test.go	8 parity scenarios with appropriate epsilon tolerance; boundary zone handling for nodes near MinActivation threshold is well-reasoned.

Sequence Diagram

sequenceDiagram
    participant H as handler_active
    participant NE as NativeEngine
    participant S as Store (Multi/SQLite)
    participant FF as sproink FFI (Rust)

    H->>NE: Activate(ctx, seeds)
    NE->>S: Version() [atomic]
    alt "store version > engine version"
        NE->>NE: Rebuild(ctx) [write lock]
        NE->>S: Version() [snapVersion]
        NE->>S: GetAllEdges(ctx)
        S-->>NE: []Edge (local + global)
        NE->>S: "QueryNodes(kind=behavior)"
        S-->>NE: []Node (isolated nodes)
        NE->>FF: sproink_graph_build(numNodes, edges, weights, kinds)
        FF-->>NE: "*SproinkGraph (CSR)"
        NE->>NE: atomic.Store(version, snapVersion)
    end
    NE->>NE: RLock
    NE->>NE: map seeds → u32 via IDMap
    NE->>FF: sproink_activate(graph, seeds, params)
    FF-->>NE: "*SproinkResults"
    NE->>NE: map u32 results → UUID via IDMap
    NE->>NE: RUnlock
    NE-->>H: []Result (sorted by activation desc)
    H->>S: TouchEdges + Hebbian learning (Go-side)

Loading

_{Reviews (20): Last reviewed commit: "fix(test): stabilize hybrid scorer tests..." | Re-trigger Greptile}

[greptile-apps]

Two issues found — one correctness bug, one dead code cleanup.

P1 — TOCTOU in NewNativeEngine (same class of bug as the one fixed in Rebuild): The Rebuild method correctly captures snapVersion before calling loadGraph, but NewNativeEngine still calls s.Version() after loadGraph returns. If a concurrent write lands between GetAllEdges inside loadGraph and the s.Version() call, the engine is initialised with a stale graph but a version counter that already reflects the post-write state. The next Activate call will then observe store.Version() <= e.version and skip the rebuild, silently serving a stale graph until another write arrives.

P2 — Dead else-branch in activator selection: *MultiGraphStore now implements ExtendedGraphStore in full (it gained GetAllEdges and Version() in this PR). The store.GraphStore(graphStore).(store.ExtendedGraphStore) assertion will therefore always succeed, making the else branch (activator = spreading.NewEngine(graphStore, spreadConfig)) unreachable dead code. The intermediate store.GraphStore(...) widening conversion is also redundant.