LDA Multicore Windows Error

[hiral2cool]

os : windows 10
version : Python 2.7.10
all latest numpy,scipy everything upgraded

when i start ldamulticore error is like this

PYTHON

lda = gensim.models.LdaMulticore(corpus=corpus, id2word=id2word, num_topics=10)

ERROR

File "C:\Python27\lib\multiprocessing\forking.py", line 361, in get_command_line
is not going to be frozen to produce a Windows executable.''')
RuntimeError:
Attempt to start a new process before the current process
has finished its bootstrapping phase.

        This probably means that you are on Windows and you have
        forgotten to use the proper idiom in the main module:

            if __name__ == '__main__':
                freeze_support()
                ...

        The "freeze_support()" line can be omitted if the program
        is not going to be frozen to produce a Windows executable.

working perfect on Ubuntu but problem in windows version
any help will be useful

[tmylk]

Does anyone have windows 10 to try if adding if __name__ == '__main__': helps as in SO?

[hiral2cool]

but at where i have to add this code ??
first i try to add this in main my python code file from which i am going to run LDA module

if name == 'main':
freeze_support()

i try this but same error

[tmylk]

It should be in the main module of your program as in https://docs.python.org/dev/library/multiprocessing.html#multiprocessing.freeze_support

May I ask for the full code that reproduces the error?

[hiral2cool]

Here i put all the files of
LDA multicore , Debug Console , Log files after execution

http://www.pastefile.com/hkNFWB

LDA single core working fine
but LDA multicore and LDA distributed both have same error and same problem

i also notice that
worker = 3
Queue size is = 3 x 2 = 6
and chunksize = 2000

so up to 5

2016-10-14 10:59:37,586 : INFO : PROGRESS: pass 0, dispatched chunk #1 = documents up to #4000/18206, outstanding queue size 2
2016-10-14 10:59:40,446 : INFO : PROGRESS: pass 0, dispatched chunk #2 = documents up to #6000/18206, outstanding queue size 3
2016-10-14 10:59:43,378 : INFO : PROGRESS: pass 0, dispatched chunk #3 = documents up to #8000/18206, outstanding queue size 4
2016-10-14 10:59:45,845 : INFO : PROGRESS: pass 0, dispatched chunk #4 = documents up to #10000/18206, outstanding queue size 5

then error ....

[tmylk]

Hmm probably if name == 'main': freeze_support() should be on the top of code.py, straight after imports?
You can use gist for uploading files

[hiral2cool]

yes i tried just now & everything is same
console output , log file ,error
everything same no change

don't know what is the problem i also try to debug code but not understand well

[tmylk]

hmm maybe that line needs to be in LDAMultiCore?
This is not a specifically gensim issue though. It relates to forking on windows.

[hiral2cool]

yup i know that it is problem of forking on windows
but it should work and something somewhere this library code will change to support new win 10 os

and i also try LDAMulticore
same issue remaining with same errors

[hiral2cool]

adding this to main source code
if name == 'main':

run working of program successfully LDAmulticore
so Issue Fixed..
Thank you

[tmylk]

@hiral2cool Thanks a lot for reporting your fix!
Which file was it added to?
May I kindly ask you to submit a Pull Request so that the community can benefit from your solution.

[tmylk]

Ping @hiral2cool could you copy paste your fixed code here?

[hiral2cool]

yaa sure here is my code
no need to change any files in gensim
just simply add line in main source code where parallel starts

http://www.pastefile.com/PZKSa3

but may be sometimes problem occurs with this is
this error may come many times

forrtl: error (200): program aborting due to control-C event

but we try to run again then also sometimes not come and run program successfully

[tmylk]

Resolved by adding this code:

if __name__ == '__main__':
    freeze_support()
    lda = gensim.models.LdaMulticore(corpus=corpus, id2word=id2word, num_topics=4)