fix: limit depth of index discovery to prevent slow scans#196
fix: limit depth of index discovery to prevent slow scans#196majiayu000 wants to merge 2 commits intoyichuan-w:mainfrom
Conversation
Fixes yichuan-w#122 The `leann list` command was scanning entire directory trees using `rglob()`, causing extremely slow performance when run in large directories like $HOME. Changes: - Add `_find_meta_files_limited()` method with max_depth parameter - Skip common large directories (node_modules, .venv, .git, etc.) - Apply limited search in `_discover_indexes_in_project()` and `_find_all_matching_indexes()` - Add `_has_app_indexes_limited()` in registry.py for faster checks - Add comprehensive tests for the new functionality Signed-off-by: majiayu000 <1835304752@qq.com>
|
Thanks for working on this! A few suggestions: 1. Add
|
Address reviewer feedback by making the directory scan depth configurable instead of hardcoding it to 3. Users with deeply nested project structures can now increase the depth limit as needed. - Add --max-depth argument to list command (default: 3) - Update list_indexes() and _discover_indexes_in_project() to accept max_depth - Add tests for the new CLI option and custom depth behavior 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Thanks for the feedback @andylizf! I've added the # Default depth (3)
leann list
# Scan deeper directories
leann list --max-depth 5For the global registry suggestion (making |
a8e6f17 to
ba91644
Compare
|
Update: I've also implemented the long-term global registry solution in a separate PR #199.
This PR can be merged independently. PR #199 builds on top of this one. |
, yichuan-w#196, yichuan-w#199 - Fix unused variable `zmq_port` in warmup() method (F841) - Remove unused imports in test_cli_list_performance.py (F401) - Sort imports in test files (I001) - Fix test_warmup.py: correct attribute name `_warmup_enabled` → `_warmup` - Add skipif for DiskANN test when backend is not installed - Apply ruff format to all modified files Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Summary
leann listscans all files in large directories like $HOME_find_meta_files_limited()method with configurable max_depth (default: 3)Fixes #122
Test plan
tests/test_cli_list_performance.py