Validator changes: always check unversioned '/versions' and handle rich HTML pages

[ml-evs]

This PR does two things:

• Some services e.g. netlify have rich 404 pages which yield unhelpful error messages, those error messages are now improved.
• Following discussion in Implement version negotiation for validator #662, this PR always checks the existence of the unversioned /versions endpoint, even when passed a versioned URL

[codecov]

Codecov Report

Merging #665 (4261f3e) into master (33de78f) will decrease coverage by 0.14%.
The diff coverage is 25.00%.

@@            Coverage Diff             @@
##           master     #665      +/-   ##
==========================================
- Coverage   93.45%   93.30%   -0.15%     
==========================================
  Files          61       61              
  Lines        3299     3300       +1     
==========================================
- Hits         3083     3079       -4     
- Misses        216      221       +5     

Flag	Coverage Δ
project	93.30% <25.00%> (-0.15%)	⬇️
validator	66.06% <16.66%> (-0.06%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
optimade/validator/validator.py	82.22% <18.18%> (-0.93%)	⬇️
optimade/validator/utils.py	94.52% <100.00%> (-0.12%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33de78f...4261f3e. Read the comment docs.

[ml-evs]

Changed to low-priority as this is only an issue in cases where the validator should fail anyway, i.e. this PR is just an error message improvement, initially we thought it was bug fix that it was failing because of this

[CasperWA]

Looks good to me, cheers @ml-evs !