The runTests helper script can be used to simplify creation and running of new tests. E.g.:
runTests gun
will read the test.cfg
configuration file where it will find the specification
for the "gun" test and it will expand it to a shell script and run it. You can
see what runTests will do by supplying the --dry-run
/ -n
option.
Each test consists of a variable number of steps, e.g.:
example:
steps:
- name: SIM
- name: REC
By default a step will expand to:
# Running Test example
cd test/<test name> && rm -rf *.root *.dat *.log fort* hlt hough raw* recraw/*.root recraw/*.log
# Step: <step name>
# Variant default
aliroot -b -q <lowercase step name>.C 2>&1 | tee <lowercase step name>.log
mv syswatch.log <lowercase step name>watch.log
Where in this case:
- is
example
- is either SIM or REC
- is either
sim
orrec
Steps of a same test are guaranteed to be performed in order, as specified in the configuration file. E.g. SIM will always preceed REC for the above test.
By default the steps will do the typical aliroot macro execution, as illustrated above, simply because that's the most common usecase.
It is however possible to customize a step by supplying the following parameters:
- pre: What to execute before the step
- driver: what to execute as a given step
- post: what to execute after the step
- setup: this is a per-test customization which allows to specify what to do before all steps.
For example the fastjetload
test, also defined in test.cfg, looks like:
loadfastjet:
setup: ""
steps:
- name: LOAD
pre: "export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib:/usr/lib64"
driver: "root -l -b -q ${ALICE_PHYSICS}/PWGJE/macros/TestLoadFastJet.C 2>&1"
post: ""
This will expand to simply:
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/usr/lib:/usr/lib64 root -l -b -q ${ALICE_PHYSICS}/PWGJE/macros/TestLoadFastJet.C 2>&1
One of the main reason to have a script like runTests
, apart from easier
parsing of the test details, is to allow for what we call "variants": simple
changes of the default behavior of the test which share most of the
configuration, but then extend it in order to use special tools, for example
valgrind or igprof. Variants are defined per step and can be specified via a
set of additional configurable options:
- prefix: a prefix to the driver part.
For example if you want to define an igprof_performance and igprof_memory
variant for the gun
test you can do:
gun:
steps:
- name: SIM
variants:
default: {}
igprof_memory:
prefix: "igprof -mp -o igprof.%(test_name)s_%(step_name)s_MEMORY.gz "
igprof_performance:
prefix: "igprof -pp -o igprof.%(test_name)s_%(step_name)s_PERFORMANCE.gz "
You can specify which variant to run by passing them to runTests
via the
--variant
option as comma separated labels. E.g. test/runTests gun --variant default,igprof_memory
.
FIXME: add the ability to specify a label for a given test, so that tests can be
selected by label (e.g. run all the tests in the pullrequest
label).
FIXME: add the ability to specify "cost" of a given test, so that we can select only
those which add up to a maximum amount of time.
FIXME: "default" variant should always be there.
FIXME: "default" variant should be run if the requested variant is not found in
a given step.
FIXME: add ability to have a step depend on the step of a different test, for example
have a "PbPbsimulation" test which has a SIM step and then have multiple "XYZreco"
tests which have a REC step which depends on "PbPbsimulation/SIM" so that simulation
is done only once.