Fix: Delay in Image Visibility After Adding Large Folder (#591)

[Akash1xe]

Fix for Issue #591 — Delay in Image Visibility After Adding Folder

fixes #591

When a Large folder with many images (~200) is added, images appear on the home tab only after all thumbnails are generated and inserted in the database. This created a visible delay of ~8 seconds.

✔ What I changed:

• Modified the folder-processing workflow to emit incremental updates instead of waiting for full completion.
• Frontend now receives image updates immediately and displays them progressively.

🔍 How I tested:

• Added a folder with 200+ mixed-size images.
• Verified that images stream one-by-one.
• Confirmed there is no freeze or UI block.

🚀 Result:

Images now appear gradually and the user no longer experiences long delays.

Please review and let me know if any adjustments are needed.

Summary by CodeRabbit

• New Features

  • Added automatic image count polling after folder addition for real-time updates.
  • New pending state indicator when adding folders.

• Bug Fixes

  • Improved error handling to continue processing if a folder encounters an issue.

• Performance

  • Optimized image processing with batch operations for faster performance.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

The backend image processing refactors from bulk multi-image preparation to per-image preparation with batch insertion, committing to the database every 50 records. The frontend adds polling to track image count updates after folder addition, exposing an isAddingFolder state. A public function signature changes from accepting multiple images to processing single images.

Changes

Cohort / File(s)	Change Summary
Backend image processing backend/app/utils/images.py	Refactors image_util_prepare_image_records to image_util_prepare_single_image_records, changing from multi-image list processing to per-image processing. Introduces BATCH_SIZE (50) constant and batching logic to commit records every 50 images or on flush. Adds total_processed tracking and enhanced error handling.
Frontend folder polling frontend/src/hooks/useFolder.ts	Adds isAddingFolder return property and implements polling mechanism using useQueryClient to track image count updates after successful folder addition. Polling starts on success and stops after three stable image count increments.
Frontend formatting frontend/src/api/api-functions/folders.ts, frontend/src/hooks/useFolderOperations.tsx	Adjusts trailing newlines; no functional changes.

Sequence Diagrams

sequenceDiagram
    participant Client
    participant Backend
    participant DB

    Client->>Backend: Request batch image processing
    loop Per image in folder
        Backend->>Backend: Prepare single image record
        Backend->>Backend: Add to batch accumulator
        alt Batch full (50 records)
            Backend->>DB: Bulk insert batch
            Backend->>Backend: Clear accumulator, reset count
            DB-->>Backend: Success
        end
    end
    alt Remaining records
        Backend->>DB: Flush final batch
        DB-->>Backend: Success
    end
    Backend-->>Client: Processing complete

Loading

sequenceDiagram
    participant User
    participant Frontend
    participant Backend
    participant QueryClient

    User->>Frontend: Add folder
    Frontend->>Backend: POST /add_folder
    Backend-->>Frontend: Success
    Frontend->>QueryClient: Start polling interval
    loop Every ~1 second
        Frontend->>Backend: GET image count
        Backend-->>Frontend: Current image count
        Frontend->>Frontend: Check if count stable
        alt Count stable (3 increments)
            Frontend->>QueryClient: Stop polling
            Frontend->>QueryClient: Clear interval
        end
    end
    Frontend-->>User: Folder added, images loading

Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

• backend/app/utils/images.py: Review batch commit logic, ensure batch boundaries and error handling don't lose records, verify final flush behavior
• frontend/src/hooks/useFolder.ts: Validate polling interval cleanup, confirm queryClient invalidation timing, check for memory leaks from polling intervals
• Cross-verify that batch size (50) and polling intervals provide acceptable UX without overloading database/API

Possibly related issues

• Title: Delay in Image Visibility After Adding Folder #591: The backend batching for incremental image processing and frontend polling mechanism directly address the core objective of displaying images incrementally as they are processed after folder addition.

Poem

🐰 Hop, hop, batch by batch,
Fifty images at a hatch,
Poll the count, nice and slow,
Watch those pictures start to flow!
✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main fix: addressing a delay in image visibility when adding large folders, which matches the core objective and backend/frontend changes.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (4)

backend/app/utils/images.py (2)

41-83: Batching logic looks sound; consider propagating DB insertion failures

The new batching loop (with BATCH_SIZE = 50) correctly:

• Prepares records per image,
• Flushes on batch size or last image, and
• Keeps a running total_processed count.

Two improvements worth considering:

1. Check db_bulk_insert_images result
   Right now the boolean return is ignored, so a failed insert still increments total_processed and the function ultimately returns True. If inserts can fail (e.g., DB unavailable), you may want to:

   • Log a more explicit error, and
   • Either break out and return False or at least avoid incrementing total_processed for failed batches.

2. Make BATCH_SIZE configurable (optional)
   If you anticipate very large folders or different storage backends, making BATCH_SIZE a module-level constant or reading it from config would ease tuning without code changes.

Also applies to: 93-99

---

154-188: Single-image preparation helper is clear; tighten typing and edge handling

The new image_util_prepare_single_image_records helper is straightforward and reuses existing thumbnail/metadata utilities. A couple of refinements:

• Type hints: Prefer more explicit typing, e.g.
  • Return type: Optional[Dict[str, Any]] instead of Dict | None.
  • This matches the actual payload shape and plays nicer with type checkers.
• Folder ID falsy check: if not folder_id: will treat 0 as “no folder”, even though image_util_find_folder_id_for_image is annotated to return int. If 0 can’t happen in practice that’s fine, but consider explicitly checking is None for future-proofing.

These don’t change behavior but improve static safety and clarity.

frontend/src/hooks/useFolder.ts (2)

17-21: Global polling refs work but NodeJS.Timeout type may be brittle

Storing the interval and counters at module scope is a reasonable way to keep polling alive across hook remounts and ensure only one active interval.

For stronger TS portability (browser + Tauri + Node), consider changing the interval type to:

const pollIntervalRef: { current: ReturnType<typeof setInterval> | null } = {
  current: null,
};

This avoids depending on the NodeJS namespace and still works with clearInterval.

---

26-75: Polling logic is reasonable; guard against never-ending polling if images data stays empty

The success callback and follow-up useEffect correctly:

• Immediately invalidate ['images'] to show the first batch,
• Clear any prior interval before starting a new one, and
• Stop polling after three consecutive stable (> 0) image-count readings.

Two edge cases to consider:

1. When ['images'] data shape differs or is never fetched
   You assume queryClient.getQueryData(['images']) returns something like { data: any[] }. If:

   • The query isn’t mounted yet, or
   • The data shape is different,
     currentImageCount will remain 0, so the > 0 condition never holds and stableCountIterationsRef never increments. Result: polling continues indefinitely every 2 seconds.

   You could:

   • Add a max-iterations or timeout safeguard, or
   • Explicitly bail out if imagesData is undefined for several cycles.

2. Logging-only effect
   The useEffect that logs success and clears polling on error is fine, but make sure future refactors don’t start mutating state that depends on stale closures there (right now it’s only reading global refs, so you’re safe).

Returning isAddingFolder: addFolderPending is a nice touch for the UI to show a pending state.

Also applies to: 77-90, 110-114

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c37d8df and 0463535.

📒 Files selected for processing (4)

• backend/app/utils/images.py (4 hunks)
• frontend/src/api/api-functions/folders.ts (1 hunks)
• frontend/src/hooks/useFolder.ts (3 hunks)
• frontend/src/hooks/useFolderOperations.tsx (1 hunks)

🧰 Additional context used

🧬 Code graph analysis (1)

backend/app/utils/images.py (1)

backend/app/database/images.py (1)

• db_bulk_insert_images (89-120)

🔇 Additional comments (2)

frontend/src/api/api-functions/folders.ts (1)

74-84: Formatting-only change here; behavior is unchanged

The return shape and logic of getFoldersTaggingStatus remain the same; this closing brace tweak has no functional impact.

frontend/src/hooks/useFolderOperations.tsx (1)

163-179: No functional change in default export

useFolderOperations remains both a named and default export; this line change is effectively formatting-only and doesn’t alter runtime behavior.