word embeddings from word2vec fails to load correctly #1656
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
lambdaofgod
force-pushed
the
fix_word2vec
branch
from
c725ec8 to
a0cf436
Compare
|
Hello! Thanks for your PR pointing me to this issue! I've resolved this issue by merging #1875 which also had a clean solution. As a result, I will me closing this, and the next version of Sentence Transformers should include word2vec support.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Loading from word2vec format fails if txt file contains first line with number of words and dimensionality.
For example gensim exports to .txt file with the first line like this:
In the current version loading will set dimensionality to 1 and fail to load vectors.
I fixed this by checking whether first line contains exactly two tokens, and treating the second one as dimensionality.