Skip to content

2.25.1.0-b306

Summary:
ybhnsw is yb's access method implementation of hnsw (hierarchical navigable search worlds) indexes. These indexes have a few creation parameters, `ef_construction` and `m`, whose details can be found [[ https://github.com/nmslib/hnswlib/blob/master/ALGO_PARAMS.md | here ]]. Before this change, users could create such ybhnsw indexes on vector columns as such:

`CREATE INDEX sample_vectors_ybhnsw_idx ON sample_vectors USING ybhnsw (vector_column);`

This diff adds reloptions ef_construction and m to allow users to control these parameters during ybhnsw index creation as follows:

`CREATE INDEX sample_vectors_ybhnsw_idx ON sample_vectors USING ybhnsw (vector_column) WITH (m = 32, ef_construction = 300);`

This change pushes `ef_construction` and `m`  to DocDb by adding corresponding fields to `YbcPgVectorIdxOptions` inside src/yb/yql/pggate/ybc_pg_typedefs.h and `PgVectorIdxOptionsPB` in common.proto.

This diff simply passes down these parameters to DocDB but does not change any DocDB code to use these parameters. The DocDB changes required to actually use these parameters will be done in a followup.

Note:

- a new function parameter "reloptions" is added to amapi yb_ambindschema.
- _PG_init has been moved from ivfflat.c to vector.c just like in upstream pgvector v0.8.0.

**Upgrade/Rollback safety:**
This change adds two fields to `PgVectorIdxOptionsPB` in common.proto, `hnsw_ef` and `hnsw_m`. These fields are part of an unreleased feature so it should not be used. New code to use these fields in DocDB should test to make sure it runs fine without valid values in these fields.

Test Plan: ./yb_build.sh --java-test 'org.yb.pgsql.TestPgRegressThirdPartyExtensionsPgVector'

Reviewers: jason, aleksandr.ponomarenko

Reviewed By: jason, aleksandr.ponomarenko

Subscribers: kramanathan, aleksandr.ponomarenko, mlillibridge, yql, ybase

Differential Revision: https://phorge.dev.yugabyte.com/D41658
Assets 2
Loading