Experimental: simple synonym list #581
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The synonyms are applied at query time. The list must be given at runtime and may be modified in between runs. The synonym file must be a simple text file. Each line contains a comma-separated list of mutual synonyms. Only single word synonyms are allowed.
Synonyms are applied after the lowercase filter, so make sure they are in the right format.
otbutz
reviewed
Jun 8, 2021
| @@ -72,7 +72,7 @@ public IndexSettings setSynonyms(String synonymFile) throws IOException { | |||
| if (line.indexOf(' ') >= 0) { | |||
| throw new RuntimeException("Synonym list must not contain any spaces or multi word terms."); | |||
| } | |||
| synonyms.put(line); | |||
| synonyms.put(line.toLowerCase()); | |||
There was a problem hiding this comment.
line.toLowerCase(Locale.ROOT) ?
There was a problem hiding this comment.
It should be "whatever ES uses", to be precise.
|
Now included in #585. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds a new command-line parameter
-synonym-filewhich can be used to experiment with applying synonyms at query time. The synonym list is loaded when Photon is started. It is not necessary to reindex the existing database and will work against our precomputed database dumps.The synonym file must be a simple text file. Each line contains a comma-separated list of mutual synonyms. Only single word
synonyms are allowed. To that end, the loader will reject any synonym file that contains spaces.
This PR is unlikely to make it into master as is. I just thought I put it out in public in case somebody else wants to experiment with using synonyms.
@hbruch