Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Remove multiple vectors references during flush #2207

Open
navneet1v opened this issue Oct 13, 2024 · 0 comments
Open

[Enhancement] Remove multiple vectors references during flush #2207

navneet1v opened this issue Oct 13, 2024 · 0 comments
Labels
Enhancements Increases software capabilities beyond original client specifications good first issue Good for newcomers v2.19.0

Comments

@navneet1v
Copy link
Collaborator

navneet1v commented Oct 13, 2024

Description

With upgrade to Lucene 9.12, Lucene started exposing the FlatVectorsFormat as KnnVectorsFormat. With this FlatFieldVectorsWriter now exposes DocsWithFieldSet and vectors which are added in the FlatFieldVectorsWriter during flush.

Now NativeEngineFieldVectorsWriter and FlatFieldVectorsWriter stores the same reference of vectors and docIds, which is not required. We can completely get rid of reference of vectors and docIds from NativeEngineFieldVectorsWriter and just use FlatFieldVectorsWriter during flush. This will simplify the code and will also free up some resources from heap.

@navneet1v navneet1v added the Enhancements Increases software capabilities beyond original client specifications label Oct 13, 2024
@shatejas shatejas added good first issue Good for newcomers and removed good first issue Good for newcomers labels Oct 15, 2024
@vamshin vamshin added v2.19.0 good first issue Good for newcomers and removed v2.18.0 labels Oct 31, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancements Increases software capabilities beyond original client specifications good first issue Good for newcomers v2.19.0
Projects
Status: 2.19.0
Development

No branches or pull requests

3 participants