Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for RediSearch v2.0 Index Definition #81

Merged
merged 13 commits into from
Aug 17, 2020

Conversation

filipecosta90
Copy link
Collaborator

@filipecosta90 filipecosta90 commented Aug 8, 2020

This PR follows the release of the first milestone in the development of RediSearch 2.0.
RediSearch 2.0, marks the re-architecture of the way indices are kept in sync with the data. Instead of having to write data through the index (using the FT.ADD command), RediSearch will now follow the data written in hashes and automatically index it. as follows:

image

API changes on RediSearch 2.0 that need to be reflected on the client

As noted here, RediSearch 2.0 milestone includes several changes to the API:

  • The index no longer lives in the key space. If you used the index key (idx:) to list the indexes in the database, for example, this will no longer work. However, we introduced a command FT._LIST to return all the indices in the database.
  • Indexes must be created with a prefix/filter. These specify which documents will be indexed automatically by RediSearch. You can specify a simple prefix and/or a complicated filter expression.
  • Upgrades are not possible. If you have an RDB created with an older version of RediSearch, RediSearch 2.0 will not be able to read it. Currently, you will have to re-index the entire data set. We are, however, working on an upgrade process for the GA release.
  • It works only with Redis 6 and above.
  • The FT commands were mapped to their Redis-equivalent commands. This allows existing applications to still work with RediSearch 2.0. The mappings are as follows:
        FT.ADD => HSET
        FT.DEL => DEL (DD by default)
        FT.GET => HGETALL
        FT.MGET => HGETALL
  • The inverted index itself is no longer saved to the RDB. This does not mean that persistency is not supported. RediSearch does save the index definition to the RDB and index the data in the background after Redis is started. You can find out when the reindexing is finished by checking the indexing status using the FT.INFO command.

Client Changes addressed in this PR

  • Added CreateIndexWithIndexDefinition to enable specifying a index definition for automatic indexing on Hash update
  • Include tests for the newly added API
  • Include godoc examples for the newly added API
    • Added ExampleClient_CreateIndexWithIndexDefinition
  • Deprecate the commands specified on RediSearch 2.0
    • Deprecated AddHash and made reference to HSET usage. Included example named ExampleClient_CreateIndexWithIndexDefinition
    • Deprecated SynAdd and made reference to use SynUpdate instead

@lgtm-com
Copy link

lgtm-com bot commented Aug 8, 2020

This pull request introduces 1 alert when merging 55d65fa into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@lgtm-com
Copy link

lgtm-com bot commented Aug 9, 2020

This pull request introduces 1 alert when merging 0d15c28 into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@codecov
Copy link

codecov bot commented Aug 9, 2020

Codecov Report

Merging #81 into master will increase coverage by 0.87%.
The diff coverage is 95.29%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #81      +/-   ##
==========================================
+ Coverage   76.18%   77.06%   +0.87%     
==========================================
  Files          12       13       +1     
  Lines        1205     1273      +68     
==========================================
+ Hits          918      981      +63     
- Misses        227      232       +5     
  Partials       60       60              
Impacted Files Coverage Δ
redisearch/document.go 72.58% <ø> (ø)
redisearch/query.go 86.54% <ø> (+1.14%) ⬆️
redisearch/client.go 78.97% <86.66%> (-1.19%) ⬇️
redisearch/index.go 97.14% <97.14%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 207c949...8e48b0d. Read the comment docs.

@lgtm-com
Copy link

lgtm-com bot commented Aug 9, 2020

This pull request introduces 1 alert when merging 4b79274 into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@lgtm-com
Copy link

lgtm-com bot commented Aug 9, 2020

This pull request introduces 1 alert when merging 08e9562 into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@lgtm-com
Copy link

lgtm-com bot commented Aug 9, 2020

This pull request introduces 1 alert when merging 4958ddc into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

Copy link
Collaborator

@MeirShpilraien MeirShpilraien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, small comments, let me know what you think.

@lgtm-com
Copy link

lgtm-com bot commented Aug 17, 2020

This pull request introduces 1 alert when merging eb0a92d into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@lgtm-com
Copy link

lgtm-com bot commented Aug 17, 2020

This pull request introduces 1 alert when merging d334849 into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@lgtm-com
Copy link

lgtm-com bot commented Aug 17, 2020

This pull request introduces 1 alert when merging c0188dd into 207c949 - view on LGTM.com

new alerts:

  • 1 for Useless assignment to local variable

@filipecosta90
Copy link
Collaborator Author

Looks good, small comments, let me know what you think.

Valid points :) made getRediSearchVersion internal so that we use it only for tests.
Addressed the other comments as well

Copy link
Collaborator

@MeirShpilraien MeirShpilraien left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@filipecosta90 filipecosta90 merged commit 9b11c55 into master Aug 17, 2020
@filipecosta90 filipecosta90 deleted the index.definition branch August 17, 2020 10:46
@filipecosta90 filipecosta90 changed the title [WIP] Added support for RediSearch v2.0 Index Definition Added support for RediSearch v2.0 Index Definition Aug 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants