Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes issues while loading word2vec and doc2vec models saved using old Gensim versions. Fix #2000, #1977 #2012

Merged
merged 13 commits into from
Apr 12, 2018
Merged

Fixes issues while loading word2vec and doc2vec models saved using old Gensim versions. Fix #2000, #1977 #2012

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
4edf610
adds default values for attributes
manneshiva Mar 31, 2018
dd1630c
ignore values for attributes that do not exist
manneshiva Mar 31, 2018
f8b4e32
adds unit test
manneshiva Mar 31, 2018
e09eba7
fixes default values for missing attributes for older gensim models
manneshiva Apr 1, 2018
b572679
adds unit test cases for loading really old gensim models
manneshiva Apr 1, 2018
310e652
Merge branch 'develop' of https://github.com/RaRe-Technologies/gensim…
manneshiva Apr 5, 2018
86e2575
adds test cases for loading all old models
manneshiva Apr 5, 2018
b71a8c4
adds more tests post loading
manneshiva Apr 8, 2018
dc54d38
handles loading d2v models saved using version <=0.12.2
manneshiva Apr 8, 2018
fd5e5b1
fix `max_rawint` value and PEP8 errors
manneshiva Apr 8, 2018
89edbe7
adds saving and loading back tests
manneshiva Apr 10, 2018
463f491
adds comments and fixes `max_rawint`
manneshiva Apr 10, 2018
9871304
fix PEP8
menshikh-iv Apr 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions gensim/models/deprecated/doc2vec.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,12 @@ def load_old_doc2vec(*args, **kwargs):
# set doc2vec vocabulary attributes
new_model.docvecs.doctags = old_model.docvecs.doctags
new_model.docvecs.count = old_model.docvecs.count
if hasattr(old_model.docvecs, 'max_rawint'): # `doc2vec` models before `0.12.3` do not have these 2 attributes
if hasattr(old_model.docvecs, 'max_rawint'): # `doc2vec` models before `0.12.3` do not have these 2 attributes
new_model.docvecs.max_rawint = old_model.docvecs.__dict__.get('max_rawint')
new_model.docvecs.offset2doctag = old_model.docvecs.__dict__.get('offset2doctag')
else:
new_model.docvecs.max_rawint = len(old_model.docvecs.index2doctag) if old_model.docvecs.index2doctag else old_model.docvecs.count
new_model.docvecs.max_rawint = \
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic: definitely deserves a comment.

len(old_model.docvecs.index2doctag) - 1 if old_model.docvecs.index2doctag else old_model.docvecs.count - 1
new_model.docvecs.offset2doctag = old_model.docvecs.index2doctag

new_model.train_count = old_model.__dict__.get('train_count', None)
Expand Down