check if a spider exists before schedule it

[artemdevel]

A simple check before schedule a spider.

By default it is possible to schedule anything using Scrapyd API:
curl http://localhost:6800/schedule.json -d project=default -d spider=nospider

this command returns {"status": "ok", "jobid": "455c1444a6c611e29f650800272a6d06"}
and also a misleading log like this https://gist.github.com/artem-dev/8a567c1775ab9bb8d122 will be created (this gist contains the complete log)

With this check nothing will be scheduled and the response will be like this:
{"status": "error", "message": "spider 'nospider' not found"}

[pablohoffman]

This runs get_spider_list (which is an expensive operation) on every schedule call, as opposed to just do it on deploy (addversion) calls.

I think it would be better if scrapyd caches the list of spiders somewhere, it could be on a sqlite.

[artemdevel]

yeah, agree.. I'll think about sqlite approach

[pablohoffman]

See followup #17