Skip to content

Create int bfloat16 vector formats #136627

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 22 commits into
base: lucene_snapshot
Choose a base branch
from
Open

Create int bfloat16 vector formats #136627

wants to merge 22 commits into from
+853 −2

Conversation

@thecoop
Copy link
Member

@thecoop thecoop commented Oct 15, 2025 •
edited
Loading

Create classes for quantized int vector formats that support bfloat16 and direct IO. This uses the new Lucene104 quantizer.

@elasticsearchmachine elasticsearchmachine added v9.3.0 Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch labels Oct 15, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search-relevance (Team:Search Relevance)

@thecoop thecoop changed the base branch from main to lucene_snapshot October 23, 2025 08:52
@thecoop thecoop requested a review from benwtrent October 23, 2025 14:05
@thecoop thecoop requested review from a team as code owners October 23, 2025 15:20
benwtrent
benwtrent reviewed Oct 23, 2025
View reviewed changes
Copy link
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to remember why I started that Flat vs Format pattern for bit vectors that you have continued here, I am not sure its actually warranted.

@thecoop
Copy link
Member Author

thecoop commented Oct 24, 2025

I've removed the intermediate format, and pushed everything into the top-level formats

@thecoop thecoop mentioned this pull request Oct 24, 2025
Merged
@thecoop thecoop requested a review from benwtrent October 30, 2025 09:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@benwtrent benwtrent Awaiting requested review from benwtrent

Assignees

No one assigned

Labels

>non-issue :Search Relevance/Vectors Vector search Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch v9.3.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@thecoop @elasticsearchmachine @benwtrent