[SSL: CERTIFICATE_VERIFY_FAILED] when executing python -m spacy.en.download

[jordipala]

Hi,

I am receiving the error pasted bellow when executing the command 'python -m spacy.en.download'

File "/usr/local/lib/python3.5/runpy.py", line 184, in _run_module_as_main "__main__", mod_spec) File "/usr/local/lib/python3.5/runpy.py", line 85, in _run_code exec(code, run_globals) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/spacy/en/download.py", line 13, in <module> plac.call(main) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/plac_core.py", line 328, in call cmd, result = parser.consume(arglist) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/plac_core.py", line 207, in consume return cmd, self.func(*(args + varargs + extraopts), **kwargs) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/spacy/en/download.py", line 9, in main download('en', force) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/spacy/download.py", line 24, in download package = sputnik.install(about.__title__, about.__version__, about.__models__[lang]) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/sputnik/__init__.py", line 37, in install index.update() File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/sputnik/index.py", line 84, in update index = json.load(session.open(request, 'utf8')) File "/root/.virtualenvs/jordipala_pfc/lib/python3.5/site-packages/sputnik/session.py", line 43, in open r = self.opener.open(request) File "/usr/local/lib/python3.5/urllib/request.py", line 466, in open response = self._open(req, data) File "/usr/local/lib/python3.5/urllib/request.py", line 484, in _open '_open', req) File "/usr/local/lib/python3.5/urllib/request.py", line 444, in _call_chain result = func(*args) File "/usr/local/lib/python3.5/urllib/request.py", line 1297, in https_open context=self._context, check_hostname=self._check_hostname) File "/usr/local/lib/python3.5/urllib/request.py", line 1256, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>
I've tried in three differents machines.

[Threynaud]

Same here, I've tried to download the archive and place it in the corresponding directory, hopping the command would check there before downloading it.. Is there a way to work around that issue?

[suchitagarwal]

Getting the same error, I think it is related to this:

[preziotte]

+1 Same error for me

[ines]

Alright, so this should be fixed now — try again?

We've been moving the site over to new servers and fixing the SSL certificate to finally address the model downloading issues. The problem was that @honnibal was using CloudFlare's "flexible" SSL option. We now got rid of that and installed a certificate from Let's Encrypt (which is great, btw!)

Make sure to flush your DNS cache before you reload and try again:
In Chrome: chrome://net-internals/#dns
On Mac / OSX 10.11+: sudo dscacheutil -flushcache

Alternatively, you can also use Google Public DNS, which has already updated. That's also what we used for debugging internally for the past few hours. Unfortunately, the DNS is still propagating and seems to be taking forever (see here for the current status).

[suchitagarwal]

👍

[jordipala]

All working fine. Thank you 👍

On 26 September 2016 at 23:17, Ines Montani [email protected]
wrote:

Alright, so this should be fixed now — try again?

We've been moving the site over to new servers and fixing the SSL
certificate to finally address the model downloading issues. The problem
was that @honnibal https://github.com/honnibal was using CloudFlare's
"flexible" SSL option. We now got rid of that and installed a certificate
from Let's Encrypt https://letsencrypt.org (which is great, btw!)

Make sure to flush your DNS cache before you reload and try again:
In Chrome: chrome://net-internals/#dns
On Mac / OSX 10.11+: sudo dscacheutil -flushcache

Alternatively, you can also use Google Public DNS
https://developers.google.com/speed/public-dns/, which has already
updated. That's also what we used for debugging internally for the past few
hours. Unfortunately, the DNS is still propagating and seems to be taking
forever (see here http://viewdns.info/propagation/?domain=spacy.io for
the current status).

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#507 (comment), or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAYlPCzsZX44hz6cdtBmBb0vySrSS25fks5quDZkgaJpZM4KG0Dc
.

[timohausmann]

I still get the error ... how can this be :(
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:581)>

[honnibal]

What version are you using?

[timohausmann]

I'm using Python 2.7.9, pip 9.0.1 and spacy 1.3.0

[timohausmann]

I tried again today, and this time it worked!
I changed nothing. No idea what's going on 👍 I'm just happy I can get started now.

[zang0]

I'm getting the same thing using py 2.7.12 and pip 9.0.1. I've tried all the remedies above, to no avail. Any other ideas? Error below.

Many thanks for any help.

python -m spacy.en.download
Downloading parsing model
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"main", fname, loader, pkg_name)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/spacy/en/download.py", line 25, in
plac.call(main)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/plac_core.py", line 328, in call
cmd, result = parser.consume(arglist)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/plac_core.py", line 207, in consume
return cmd, self.func(*(args + varargs + extraopts), **kwargs)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/spacy/en/download.py", line 18, in main
download('en', force=False, data_path=data_path)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/spacy/download.py", line 36, in download
about.models.get(lang, lang), data_path)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sputnik/init.py", line 37, in install
index.update()
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sputnik/index.py", line 84, in update
index = json.load(session.open(request, 'utf8'))
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/sputnik/session.py", line 43, in open
r = self.opener.open(request)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 429, in open
response = self._open(req, data)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 447, in _open
'_open', req)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain
result = func(*args)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1241, in https_open
context=self._context)
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1198, in do_open
raise URLError(err)
urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)>

[rebera]

I have the same issue / similar issue.
I can access https://index.spacy.io/models/en-1.1.0/archive.gz in the browser.
Installation of spacy works.
Download of model fails
I am behind a proxy.

Python 2.7.13
Spacy 1.6

[ines]

@rebera It's possible that the proxy is the issue here – we'll investigate. In general, problems like these should all be fixed with the v2.0 release.

We'll be able to ship much smaller models as binaries along with the release and retire the current model download process. This should also give you more control over how you want to handle the model downloads.

[rebera]

@ines Thanks for the fast reply. Is there a way to download the for the language package and locally install it until V2.0 is available?

[maksymx]

I got the same error with latest spacy version, latest pip and python 3.6.0

[ashishsingal1]

Getting a similar (but not same) issue that I've documented on SO ..

http://stackoverflow.com/questions/41725166/installing-spacy-ssl-certificate-error

Instructions for a workaround would be appreciated until the fix is pushed (i.e downloading the tar file through a URL, unzipping, etc manually).

[ghost]

Same issue.

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

-on python-3.6.0 and pip3

[maksymx]

This issue can be fixed by this walkthrough (it's for Mac):
http://stackoverflow.com/questions/41691327/ssl-sslerror-ssl-certificate-verify-failed-certificate-verify-failed-ssl-c/41692664

but in general - it's simply to update certifi package via pip

[ghost]

@yakudzam Fixed, Thanks!

[ozone3]

Hi,
I am using python 2.7.13 and Red Hat Linux, and I was also trying to download the models for spacy v. 1.6 from behind a proxy, and also got the message: "urllib2.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)." Could you provide more details on the certifi workaround or provide more details on how one could download the models manually? Also, could you give us a sense when spaCy version version 2.0 is coming out? Thanks!

[maksymx]

@ozone3 try to update certifi package. If this solution wont help - try to look into python and ssl related questions on stackoverflow.com

[ines]

@yakudzam Thanks for sharing your solution!

The Python/SSL thing really seems to be a recurring source of problems and unfortunately, there's currently no easy way for us to provide a link to download the models from the browser. So I can't wait to finally release 2.0 and retire the sputnik package manager for good.

As I said above, the plan is to attach the models as binary data to the GitHub releases. The models will also be significantly smaller.

[AliFlux]

Hi,
The issue is coming back again. Here's the log:

C:\>python -m spacy.en.download all
Downloading parsing model
Traceback (most recent call last):
	File "C:\Program Files\Python35\lib\urllib\request.py", line 1254, in do_open
		h.request(req.get_method(), req.selector, req.data, headers)
	File "C:\Program Files\Python35\lib\http\client.py", line 1106, in request
		self._send_request(method, url, body, headers)
	File "C:\Program Files\Python35\lib\http\client.py", line 1151, in _send_request
		self.endheaders(body)
	File "C:\Program Files\Python35\lib\http\client.py", line 1102, in endheaders
		self._send_output(message_body)
	File "C:\Program Files\Python35\lib\http\client.py", line 934, in _send_output
		self.send(msg)
	File "C:\Program Files\Python35\lib\http\client.py", line 877, in send
		self.connect()
	File "C:\Program Files\Python35\lib\http\client.py", line 1260, in connect
		server_hostname=server_hostname)
	File "C:\Program Files\Python35\lib\ssl.py", line 377, in wrap_socket
		_context=self)
	File "C:\Program Files\Python35\lib\ssl.py", line 752, in __init__
		self.do_handshake()
	File "C:\Program Files\Python35\lib\ssl.py", line 988, in do_handshake
		self._sslobj.do_handshake()
	File "C:\Program Files\Python35\lib\ssl.py", line 633, in do_handshake
		self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
	File "C:\Program Files\Python35\lib\runpy.py", line 184, in _run_module_as_main
		"__main__", mod_spec)
	File "C:\Program Files\Python35\lib\runpy.py", line 85, in _run_code
		exec(code, run_globals)
	File "C:\Program Files\Python35\lib\site-packages\spacy\en\download.py", line 25, in <module>
		plac.call(main)
	File "C:\Program Files\Python35\lib\site-packages\plac_core.py", line 328, in call
		cmd, result = parser.consume(arglist)
	File "C:\Program Files\Python35\lib\site-packages\plac_core.py", line 207, in consume
		return cmd, self.func(*(args + varargs + extraopts), **kwargs)
	File "C:\Program Files\Python35\lib\site-packages\spacy\en\download.py", line 18, in main
		download('en', force=False, data_path=data_path)
	File "C:\Program Files\Python35\lib\site-packages\spacy\download.py", line 36, in download
		about.__models__.get(lang, lang), data_path)
	File "C:\Program Files\Python35\lib\site-packages\sputnik\__init__.py", line 37, in install
		index.update()
	File "C:\Program Files\Python35\lib\site-packages\sputnik\index.py", line 84, in update
		index = json.load(session.open(request, 'utf8'))
	File "C:\Program Files\Python35\lib\site-packages\sputnik\session.py", line 43, in open
		r = self.opener.open(request)
	File "C:\Program Files\Python35\lib\urllib\request.py", line 466, in open
		response = self._open(req, data)
	File "C:\Program Files\Python35\lib\urllib\request.py", line 484, in _open
		'_open', req)
	File "C:\Program Files\Python35\lib\urllib\request.py", line 444, in _call_chain
		result = func(*args)
	File "C:\Program Files\Python35\lib\urllib\request.py", line 1297, in https_open
		context=self._context, check_hostname=self._check_hostname)
	File "C:\Program Files\Python35\lib\urllib\request.py", line 1256, in do_open
		raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:645)>

@ines, is there any quick fix for it? I'm using Windows 10 with Python 3.5.2. Can I manually download models and use them?

[AliFlux]

Hey!

I was just experimenting with SSL options and eureka! Found a hacky solution.

1. Visit C:\Program Files\Python35\Lib\site-packages\sputnik or where ever the sputnik is installed.
2. Open session.py
3. Add import ssl to the imports
4. Add ssl._create_default_https_context = ssl._create_unverified_context just before the r = self.opener.open(request) statement.

Now it's going to ignore all SSL errors globally. It's still a hack and must never be used in production cases.

[ines]

@AliFlux Thanks for sharing & sorry again about this mess. But, the good news are, the models are now available to download from GitHub – either from your browser, or however else you want to do it. This also means we'll be able to retire Sputnik (and the S3 infrastructure) with the next release 🎉

Here's the info I posted in #798 and #711 the other day (forgot to update this thread as well, sorry):

After trying out different solutions, we decided to simply attach the models as archive files to the latest release. The files are still quite large (between 500 and ~700MB), but they can now be downloaded via the browser.

➡️ https://github.com/explosion/spaCy/releases/tag/v1.6.0

Here's how to install the models manually:

1. Find the default data path. Use spacy.util.get_data_path() to find the directory where spaCy will look for its models, or change the default data path with spacy.util.set_data_path().
2. Unpack the archive and place the contained folder in that directory.
3. Load the model via spacy.load('en') or spacy.load('de').

cc: @jordipala, @Threynaud, @suchitagarwal, @preziotte, @timohausmann, @zang0, @rebera, @ashishsingal1, @ozone3, @DeeAnG

[AliFlux]

That's awesome! Thanks for doing this.

[rebera]

Thanks ines!

[kingchenchina]

hi, @ines i use spacy behind the proxy, and have to unpack the archive manually, but always encounter such exception:
FileNotFoundError: [Errno 2] No such file or directory: 'XXXXX/XX/XXX/en/init.py'

how to fix it?

[rishkarajgi]

python -m spacy download en_core_web_sm Traceback (most recent call last): File "/usr/lib64/python2.7/runpy.py", line 162, in _run_module_as_main "__main__", fname, loader, pkg_name) File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code exec code in run_globals File "/usr/lib64/python2.7/site-packages/spacy/__main__.py", line 31, in <module> plac.call(commands[command], sys.argv[1:]) File "/usr/lib/python2.7/site-packages/plac_core.py", line 328, in call cmd, result = parser.consume(arglist) File "/usr/lib/python2.7/site-packages/plac_core.py", line 207, in consume return cmd, self.func(*(args + varargs + extraopts), **kwargs) File "/usr/lib64/python2.7/site-packages/spacy/cli/download.py", line 28, in download shortcuts = get_json(about.__shortcuts__, "available shortcuts") File "/usr/lib64/python2.7/site-packages/spacy/cli/download.py", line 59, in get_json r = requests.get(url) File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get return request('get', url, params=params, **kwargs) File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send r = adapter.send(request, **kwargs) File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 506, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /explosion/spacy-models/master/shortcuts-v2.json (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)'),))

I am still getting this error. Please help me resolve this!

[AdeMiller]

@ines Are the models on the 1.6.0 release correct? I tried downloading the English model and unpacking it.

pwd
/home/ade/src/git/machinelearning/conga-ml-web/app/spacy
(flask) 10:14 spacy [ml/spacy-ner-test ↓·1|✚ 2…33⚑ 2] >ls -R
.:
en-1.1.0
./en-1.1.0:
deps  meta.json  ner  pos  tokenizer  vocab  wordnet
./en-1.1.0/deps:
config.json  model
./en-1.1.0/ner:
config.json  model
./en-1.1.0/pos:
model
./en-1.1.0/tokenizer:
infix.txt  morphs.json  prefix.txt  specials.json  suffix.txt
./en-1.1.0/vocab:
gazetteer.json  lemma_rules.json  lexemes.bin  oov_prob  serializer.json  strings.json  tag_map.json  vec.bin
./en-1.1.0/wordnet:
adj.exc  adv.exc  frames.vrb  index.adj  index.adv  index.noun  index.verb  noun.exc  sentidx.vrb  sents.vrb  verb.exc  verb.Framestext

Setting the data path appears to show the model with spacy info but i will not load.

>python
Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import spacy
>>> spacy.util.set_data_path("/home/ade/src/git/machinelearning/conga-ml-web/app/spacy")
>>> spacy.info()

    Info about spaCy

    Python version     2.7.12         
    Platform           Linux-4.4.0-53-generic-x86_64-with-LinuxMint-18.1-serena
    spaCy version      2.0.7          
    Location           /home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy
    Models             en-1.1.0       

>>> spacy.load("en-1.1.0")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy/__init__.py", line 19, in load
    return util.load_model(name, **overrides)
  File "/home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy/util.py", line 112, in load_model
    return load_model_from_link(name, **overrides)
  File "/home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy/util.py", line 126, in load_model_from_link
    cls = import_file(name, path)
  File "/home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy/compat.py", line 115, in import_file
    return imp.load_source(name, loc)
IOError: [Errno 2] No such file or directory
>>> spacy.load("en")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy/__init__.py", line 19, in load
    return util.load_model(name, **overrides)
  File "/home/ade/.venvs/flask/local/lib/python2.7/site-packages/spacy/util.py", line 119, in load_model
    raise IOError("Can't find model '%s'" % name)
IOError: Can't find model 'en'

[AdeMiller]

#920 (comment)

I'm still seeing this issue. The following workaround fixed it from me

pip install http://github.com/explosion/spacy-models/releases/download/en_core_web_sm-> 2.0.0/en_core_web_sm-2.0.0.tar.gz
--trusted-host github-production-release-asset-2e65be.s3.amazonaws.com -v

You may have to change the trusted host depending on what and were you are downloading. The -v > option will show the exact URL and hostname that requires a trust.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.