Skip to content

2.25.0.0-b337

Summary:
This diff introduces a Merge function of two vector indexes for HNSWlib only. It introduces an iterator over pairs <Vector, VectorId>. Into hnswlib added an iterator that uses getDataByInternalId function to obtain stored vectors. The iterator has been introduced for the hnswlib wrapper class HierarchicalNSW as well. Also there is an iterator for ShardedVectorIndex and IndexWrapperBase classes. For USearch a stub iterator has been added.
There is a limited version of the Merge function inside src/yb/vector/vectorann_util.h file. Right now it works only for small index which sum size less than 10 elements. To finalise the Merge function we need to know the size of merging indexes to estimate the size for a final merged index, right now it is a constant 10.

introduce hnswlib_index_merge-test
Jira:DB-12962

Test Plan: hnswlib_index_merge-test

Reviewers: sergei, arybochkin

Reviewed By: sergei

Subscribers: ybase

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D39478
Assets 2
Loading