Use single precision or even half precision

[roiser]

Started by OV for double -> float. Reduces register pressure. Now also typedef switch (AV).

in eemumu_AV/master

It’s a global switch, in the end one may want to have finer grained usage of single/double precision per calculation.

[roiser]

see also #6

[valassi]

I have integrated this (single precision, not half precision) here: roiser@85201ea

Here I implemented a check to avoid nans from single precision: roiser@6a31ca2

Note that FLOAT is a factor 2.3 faster than DOUBLE. I guess a factor 2 comes from the allowed use of FP32 capabilities of theGPU, the extra 15% comes from lower memory usage, possible fewer rgisters and hence better throughput?
See https://docs.google.com/document/d/1g2xwJ2FsSlxHvSUdPZjCyFW7zhsblMQ4g8UHlrkWyVw/edit#

[valassi]

Before I move to something else, I dump a few plots from the profiler here. As discussed, moving from DOUBLE to FLOAT increase throughput by more than a naive factor 2 (which one could assume because we now use the FP32 units and not only the FP64 ones). In my test it went from 6.8E8 to 1.64E9, ie a factor 2.4. For instance register usage is much smaller, and this brings a lot of benefits.

This is the overview. Registers down from 172 to 80. The number of requests is the ame but the number of transactions is divided by two as expected.

The roofline shows that FP32 is now well utilized while it was almost unused before. Conversely, FP64 is now AT ZERO.

The workload analysis shows that 32bit FMA is now the main workhorse. Instead FP64 has essentially disappeared

Memory usage mainly shows that the number of MB needed has been halved

The number of warps per scheduler more than doubvles, possible because of fewer registers used

Stall math pipe throttle more than doubles, which is good

The instruction mix sees an explosion of fp32 instructions

Finally, achieved occupancy almost triples, thanks to the reduced number of registers

Some conclusions

• this is just a feaibility study, clearly FLOAT has much better throughput than DOUBLE but physics validation is needed
• if we do investigate the FLOAT way, maybe something can be done to still keep some operations in the FP64 lines
• about validation, note that the number of distinct random numbers in [0,1] is O(E7-E8), say at least 10M, as there are ~7 significant digits; however each particle has 4 random numbers, so there are at least E28 distinct states of one particle, and there are at least four particles, so at least E104 distinct events, so this is not a problem (and along these lines one may indeed consider half precision too, at least on this specific issue?)

I would keep it to this for the moment

[valassi]

For lack of a better ticket (should open a 'numerical precision saga' with nans fast math etc) note the interesting observations in #417 (comment) when comparing fortran and cudacpp: there may be large differences, which are bigger in float than in double, but the funny thing is that cudacpp seems to be systematically below fortran (at least, the outliers are more in that direction)