First attemp on Template and Bootstrap UI support

[eLRuLL]

Hi,

I am trying to add WEB UI support for the Scrapyd service.

This is a first attemp, so I think this shouldn't be merge as it is. I want to explain first the things I am trying to do, and also ask what to do next.

• First, I saw that you were using plain string inside the website.py to populate the pages on the scrapyd service, so I am trying to use Jinja2 to better use templates (like django) to create separate html files for those pages.
• I added bootstrap 3.1.1 to a static/ folder, so we can use that css framework. I think this could also be achieved directly pointing to the css an js files on an external server, so please let me know what could be the best approach here.
• A folder templates/ has been created, there should be all the html files for the website, like home, jobs, available projects, etc. Now I have there the home.jinja file, that is intended to show the welcome page when you go to http://localhost:6800/

QUESTION: Could someone help me on how to upload or add static when you install scrapyd?
So my intention is that every installation should contain its own templates/ and static/folder. I read that it should be done on the MANIFEST.in file, but I am really lost there.

It shouldn't be so difficult, could someone maybe share what lines should go there? or maybe somewhere else?

EDIT:

• The static files have been added, and now it has support for css an js files.
• I also added the jinja2 requirement on the setup.py file.

Thank you.

[eLRuLL]

I will be adding the entire support for the different pages like jobs/ items/ logs/ ,etc.

[nyov]

There already is a fancy try at a webfrontend for scrapyd at #15
Now I don't know why that didn't make it (or why there was an idea for this in the gsoc wiki while we still have that PR open).

If I can make a suggestion here it would be this: let's NOT hack a clunky jinja-jQuery-Bootstrap frontend into scrapyd.
Keep the design separate from the service, as in, add methods to the JSONP API where necessary.
Then we can have a frontend nicely separated and your fancy javascript pages can call the JSON API. And others can use a different interface (say, a CLI or greasemonkey script) without putting all the clutter in scrapyd.

@pablohoffman, @dangra, what's your take on this?
I'd prefer to see this as a REST API that supports X(HT)ML and JSON/P for all endpoints, where you can browse the XML pages directly and use the JSON variant for scripting.
I'm ok with jinja for these templates, and some basic css, but everything else could go into a scrapyd-frontend project IMO.
(So I think #15 #14 went in the right direction, but could be stripped down somewhat.)

[eLRuLL]

I could not use bootstrap, and maybe create some css and js files just for this scrapyd service, so we don't have to depend on others.

If you feel that boostrap could be added, I am also ok to take it apart.

[Digenis]

Closing because of merge conflicts.
Don't hesitate to reopen if you are still interested.

[jpmckinney]

Spirit continued in #304