feat: Add server-side search and pagination performance improvements

[cbcoutinho]

Summary

This PR introduces server-side search functionality and several performance improvements to the Notes app, particularly for users with large note collections. The changes focus on optimizing the initial load time and implementing efficient pagination.

Related Issues

Fixes #1226 - Notes are not loading (user has ~5k notes). HTTP 504 Gateway Timeout
Addresses #966 - Notes search is very slow via global search
Possibly fixes #1407 - On web editor, notes loading forever and impossible to show/edit

Changes

Performance Improvements

• Optimize initial notes loading by excluding content field when not needed, reducing payload size
• Implement incremental pagination for notes list scrolling to load notes in chunks
• Implement chunked API loading to prevent fetching all notes on scroll
• Fix rapid-fire pagination causing all notes to load unnecessarily

Server-Side Search

• Add server-side search with pagination support allowing filtering notes by title on the backend
• Search is case-insensitive and integrated with the existing pagination system
• Includes category statistics and total count headers for better UI feedback

Bug Fixes

• Return full note metadata for pruned notes in paginated API instead of just IDs
• Prevent periodic refresh from overwriting search results to maintain UX during active searches
• Add comprehensive logging for debugging pagination issues

Modified Files

• lib/Controller/Helper.php - Added search parameter and filtering logic
• lib/Controller/NotesApiController.php - Integrated search with pagination, added category stats headers
• src/App.vue - Updated note loading logic
• src/NotesService.js - Enhanced service layer for pagination and search
• src/components/NoteRich.vue - UI adjustments
• src/components/NotesView.vue - Scroll-based pagination handling
• src/store/notes.js - State management for paginated notes
• src/store/sync.js - Sync improvements

Testing

Tested with large note collections (1000+ notes) to ensure:

• Initial load is significantly faster
• Pagination works smoothly during scrolling
• Search filters work correctly with pagination
• No duplicate notes or missing notes during pagination
• Addresses the 504 Gateway Timeout issue reported in Notes are not loading (user has ~5k notes). HTTP 504 Gateway Timeout. #1226 for users with 5000+ notes

---

This PR was generated with the help of AI, and reviewed by a Human

[cbcoutinho]

Hi friendly ping @enjeck @JuliaKirschenheuter is there interest in this PR?

[enjeck]

Thanks for working on this!
I tested locally and it does improve speed. I made a script to create 4000 notes:

#!/usr/bin/env zsh

set -u
NC_BASE_URL="http://nextcloud.local/index.php" 
NC_USER="admin"
NC_APP_PASSWORD="admin"
NOTES_CATEGORY="${NOTES_CATEGORY:-NotesTest}"
TOTAL_NOTES="${TOTAL_NOTES:-4000}"
echo "Starting..."

# Function to create a single note
create_note() {
  local idx="$1"
  local title="AutoTest Note ${idx}"
  local content="This is autogenerated content for note ${idx}.\nSeed: $(date +%s)-${RANDOM}"

  # Use a small JSON payload; favorite=false and modified=0 omit extra work
  curl --connect-timeout 5 --max-time 10 --retry 2 --retry-delay 1 -sS -u "${NC_USER}:${NC_APP_PASSWORD}" \
    -H "Content-Type: application/json" \
    -X POST \
    -d "{\"category\":\"${NOTES_CATEGORY}\",\"title\":\"${title}\",\"content\":\"${content}\"}" \
    "${NC_BASE_URL}/apps/notes/api/v1/notes" >/dev/null 2>&1 || true
}

progress_interval=100
created=0
for (( i=1; i<=TOTAL_NOTES; i++ )); do
  create_note "${i}" >/dev/null
  (( created++ ))
  if (( created % progress_interval == 0 )); then
    echo "Created ${created} notes..."
  fi
done

echo "Done. Created ${TOTAL_NOTES} notes in category '${NOTES_CATEGORY}'."

Without these changes, it take about 12 seconds for the initial page load. With this, that time is halved

[enjeck]

When the search text is cleared, we should see the list of all notes. Currently, when that happens, it's empty

[enjeck]

Plus, why have we removed the separation of notes into "days" like "Today", "Yesterday"? Now, we don't have it:

[cbcoutinho]

Great point, this was an oversight - I will try to bring those subheaders back

[enjeck]

Without these changes, it take about 12 seconds for the initial page load. With this, that time is halved

Actually, I take this back. I tested again and there doesn't seem to be any consistent speedup. What sort of testing did you do on your side to show that these changes improve performance?

[cbcoutinho]

Hi @enjeck - I tested this locally by creating 6k notes and then testing how long it took to trigger the next-page scroll capability. I didn't compare initial page load in isolation - only the time it takes to scroll through successive pages. On large numbers of notes, the initial page load would timeout without this change. I will try to gather some more profiling information and get back to you