CMake target for each accelerator

[psychocoderHPC]

Based on this I suggest that we add CMake target for each accelerator backend.
Currently alpaka CMake behaves to optimal fits the CI workflow and is activating all available backends.
Backends should be select able during the install of alpaka and store the dependencies including there locations. After alpaka is installed e.g. for CUDA than there is no requirement to search for a CUDA library again because dependencies must be checked during the install or when add_subdirectory is called.

[j-stephan]

After alpaka is installed e.g. for CUDA than there is no requirement to search for a CUDA library again because dependencies must be checked during the install or when add_subdirectory is called.

What if I don't have CUDA installed at first but install it after alpaka? Do we want to force a reinstall?

[psychocoderHPC]

Than you can not use CUDA. If you install MPI before you install CUDA you can not have a CUDA-aware MPI.
If you activate CUDA after you installed alpaka you can not guarantee that you alpaka works fine with CUDA.
The general workflow to install software is:

- configure
- make
- make test
- make install

To switch the CUDA version after installing alpaka sound good but will avoid that you are on a reproducible environment.

[psychocoderHPC]

You sill have the possibility to use alpaka via add_subdirectory to be able to switch accelerator backends.

[j-stephan]

But CUDA is a build dependency for CUDA-aware MPI, right? This isn't true for alpaka since we are header-only.

I'm also not convinced that there is a problem with our current approach. Right now we check for dependencies at build / installation / add_subdirectory as well as find_package (i.e. after installing alpaka). The latter comes with virtually no overhead. Why should we remove it?

[psychocoderHPC]

With the current approche the admin who provides alpaka as module has hard times to guarantee that alpaka works because what 'make test' checks is not what the iser later uses as environment. We should discuss it with more alpaka devs in a face to face meeting. It is hard for me to explain here all because I need maybe intruduce how HPC system handle there software and how they IMO should do it without moving software/environment tests to there users.

[psychocoderHPC]

A possible way would be to keep the live search in cmake after the installation but only allow backends those was activated and therefore could be tested during the install and disable all other.
This would still allow you to switch e.g. between different cuda versions if cuda was enabled during the install.

[ax3l]

You both have great points and I think keeping the installed version as @j-stephan describes agnostic is quite good, especially for other package managers than Spack that live with a more "runtime addon" philosophy (debian, conda-forge et al.).

The point that @psychocoderHPC raises is really good and I would say we could address this with:

• additional documentation how to test backends at install-time (mainly for sysadmins and packagers)
• additional, small test integration tests that we install into $PREFIX/shared/alpaka/examples/ that can be built to test an installed alpaka with mini-examples against various, pre-selected backends

[SimeonEhrig]

I agree with @psychocoderHPC that we should not activate all possible backends by default. It is not the usual workflow and it is a source of unnecessary bugs (e.g. In my case, OpenMP and the nvcc together caused a problem and did not use OpenMP in my code).
But I think we should not restrict the backends at install time without a technical reason. How @ax3l already mention, there are workflows, which allows to extend the environment during the development.

[j-stephan]

@SimeonEhrig has unconvered a problem with the current approach while working on vikunja. Currently we don't use CMake's install(EXPORT ...) functionality. This means that the alpaka target is not known to CMake's installation routines and we end up with a FindAlpaka.cmake that basically works as a slightly different CMakeLists.txt. While this has the benefit of being able to reconfigure alpaka after installing it, third-party libraries depending on alpaka cannot be installed anymore if they use install(EXPORT):

CMake Error: install(EXPORT "vikunjaTargets" ...) includes target "vikunja" which requires target "alpaka" that is not in any export set.

I believe we could fix this by using install(EXPORT) in alpaka's CMakeLists.txt. This will make the alpaka target known to the installation procedure and generate a FindAlpaka.cmake that has a fixed set of compiler and linker flags, definitions, include directories, libraries - so alpaka won't be reconfigurable after installation (except by reinstallation).

Unless there are any objections I'd start working on that in the coming days.

[SimeonEhrig]

In the video conference with HZB today, they explained that they want to deploy a binary with different, activated backends. At runtime, the appropriate implementation should then be selected depending on the available hardware.
To do this, the code must be compiled ntimes in n shared library each with a different active backend. At runtime, the correct one can be loaded. This is possible with the current CMake, but some combinations, such as OpenMP + Clang-CUDA, are not working. The problem would be solved if not every target would be compiled with all compiler flags for all backends, but only with the one necessary for the backend.

[fwyzard]

This is very similar to what how we use alpaka in the CMS software (without CMake, though).
Is there anything (not CMake-related) I can help with?

[SimeonEhrig]

This is very similar to what how we use alpaka in the CMS software (without CMake, though). Is there anything (not CMake-related) I can help with?

I know. Therefore I told to the HZB, that is possible ;-) Thanks for the offer.