v0.4.1 Support Append in Vector Search
The vector search in Lance now supports live updates. Previously, when you added new vectors to the dataset, you would be required to rebuild the index. Now, the index is "inherited" and the vector search results are the combination of ANN search on the indexed data and KNN on the new Appended data. So there's a small latency increase and the recall should be the same or better.
This provides a smooth performance curve until you have inserted enough new data that re-indexing is warranted.
What's Changed
- Adding secret to publish task by @gsilvestrin in #742
- [Rust] make distance function to take slice instead of Float32Array by @eddyxu in #748
- Vector search should support appending new rows by @changhiskhan in #593
- windows lapack support by @gsilvestrin in #743
- Fix LanceDataset.to_batches by @changhiskhan in #751
Full Changelog: v0.4.0...v0.4.1