Break requirements down on per backend basis

[ml-evs]

This PR splits the requirements file further for each backend (e.g. mongo, django and hopefully elasticsearch soon). This should make it easier for people to add more backends in the future, and relates to #63 which introduces additional backend-dependent code (like separate grammars).

• Each extra set of deps is grabbed automatically in setup.py based on the contents of the requirements/*.txt files.
• If a backend is not installed, the unit tests will be skipped. As mongo is currently required for the overall package, these are not skipped (though at some point we could just switch to mongomock as the only dep for this, or try to generalise the server for all backends).

[codecov]

Codecov Report

Merging #64 into master will decrease coverage by 0.12%.
The diff coverage is 71.42%.

@@            Coverage Diff             @@
##           master      #64      +/-   ##
==========================================
- Coverage    78.3%   78.17%   -0.13%     
==========================================
  Files          21       21              
  Lines         719      724       +5     
==========================================
+ Hits          563      566       +3     
- Misses        156      158       +2

Impacted Files	Coverage Δ
optimade/filtertransformers/tests/test_django.py	88.23% <71.42%> (-11.77%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 72e3411...dec3d47. Read the comment docs.

[ltalirz]

Thanks @ml-evs !

I thought the idea was to have in the setup.py more "lax" requirements (>= or even without version specification), while the requirements.txt files contain fixed dependencies that are used for testing and are guaranteed to work.

This change would remove the two types of requirements - and in that case, I no longer see a reason to have the .txt files; we could just write everything inside the setup.py.

I have to say that I was anyhow not fully convinced by this approach - if one of the newer dependencies does break optimade-python-tools, then users have to know that there are also the more strict requirements.txt files.

How about an intermediate solution, where we don't totally hardcode the version of every dependency but rather hope that they do semantic versioning correctly, i.e. something like
lark-parser>=0.5.6,<0.6
mongomock>=3.16.0,<4
(and only keep this in the setup.py)?

[ml-evs]

Hey @ltalirz, I'm happy to revert the changes to setup.py and hardcode the split requirements, I guess I normally work with fixed version reqs but can see the reasons others might not want to.

I think requirements files are still useful so the user can use their package manager of choice (e.g. conda install --file or pip install -r) separate from the setup.py.

[ltalirz]

travis fails - let me know when you want me to review;
the changes look fine to me

[ml-evs]

Ah, thought pip could figure out [all] on its own, but guess not. Should be fixed now.

[ml-evs]

We could consider switching to Pipfile only for our reqs, as FastAPI have done.

[ltalirz]

thanks @ml-evs , looks good to me

just to understand: what would be needed to be able to remove mongogrant from the default dependencies (which is currently duplicated in the mongo dependency)?

also, would vote to move to pipenv

[ml-evs]

thanks @ml-evs , looks good to me

just to understand: what would be needed to be able to remove mongogrant from the default dependencies (which is currently duplicated in the mongo dependency)?

also, would vote to move to pipenv

Nothing! :)

I'm not sure we're using it explicitly in the code, but its a useful tool either way if you're setting up a mongo to use with optimade.

+1 for pipenv, perhaps that's the next (boring) step after we get #63 merged.

[ltalirz]

thanks!

[ml-evs]

Closes #62