Write docs how hpcmp2 should be tested

[certik]

Include the following information.

One has to do several different tests:

• We have to test in pure environment (for example Vagrant) with no systemwide packages installed. See if it builds. Run --check-libs, make sure it doesn't print anything. This is the obvious test.
• Test on real computer with systemwide libraries. Some package build systems behave differently when they "see" a systemwide package. Run --check-libs, make sure it doesn't print anything (=we link with our own versions of libraries). The best is to run this test twice: with systemwide libraries installed of the same version that we have and also a different version (a different systemwide library version sometimes triggers a bug in a build system).

If things look good under all 3 tests, then we can be reasonably sure, that things work.

[cekees]

Do we want to also provide the option to test each package as we build? For example the mpi4py test are run out of mpi4py/test using 'python runtests.py', which could be added to the bash_script.arch file. Numpy and pytables can be tested with 'python -c "import numpy.numpy.test()' ".

[certik]

That is a good point. The question is whether it can ever happen that we build something and it doesn't work.

I know for sure that this happens for Python, so we do have a quick (but solid) test that Python modules work at the end of the script here: https://github.com/hashdist/python-hpcmp2/blob/master/pythonConfig/bash_script.linux

As far as mp4py, numpy or pytables --- I guess we could run the test to reveal possible bugs in those packages, but it seems to me that we, as a distribution, should mainly make sure that the build was completed correctly. Whether there is an actual bug in a package, that only shows on a certain platform, that is a different issue. Maybe we could test this issue as well, I don't know.

[cekees]

Yes, it was your python bash_script that got me thinking about testing again. Certainly we don't want to force tests all the time because they can take so long, but I was thinking that for buildbots it would be nice to go ahead and run some basic tests for each package, if they're provided.