Skip to content

Remove Lance v0.30.0 vector search workaround when bug is fixed #57

@jayscambler

Description

@jayscambler

Context

Lance v0.30.0 has a confirmed bug that causes "Task was aborted" errors when performing vector search on small datasets. This is due to a Rust panic with error: "range end index 3 out of range for slice of length 0".

Bug report: lance-format/lance#2464

Current Workaround

We've implemented a workaround in contextframe/frame.py in the _knn_table() method that catches the error and returns empty results with a warning.

Action Required

When Lance releases a fix for this bug:

  1. Update the pylance dependency to the fixed version
  2. Remove the try-catch workaround in _knn_table() method
  3. Re-enable the skipped vector search tests:
    • test_knn_search_basic
    • test_search_result_scores
    • test_empty_vector_handling

Files to Update

  • contextframe/frame.py (remove workaround in _knn_table())
  • tests/integration/test_vector_search.py (remove @pytest.mark.skip decorators)
  • pyproject.toml (update pylance version requirement)

Notes

  • The bug affects vector search on datasets with fewer than ~10 rows
  • Full-text search test is skipped for a different reason (requires inverted index)
  • Other vector search tests pass without issues

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions