Introducing SYCL implementations/variants #64
Conversation
MichaelSt98
force-pushed
the
nams_sycl
branch
from
c38cf1a to
a471d1c
Compare
| * which can be obtained at http://www.apache.org/licenses/LICENSE-2.0. | ||
| * In applying this licence, ECMWF does not waive the privileges and immunities | ||
| * granted to it by virtue of its status as an intergovernmental organisation | ||
| * nor does it submit to any jurisdiction. |
There was a problem hiding this comment.
Looks like you accidentally dropped the header in this copy-paste. This needs to go back, I think.
SYCL-specific env and README entry
bundle.yml
Outdated
| help: Enable GPU kernel variant based on SYCL | ||
| cmake: > | ||
| ENABLE_SYCL=ON | ||
| ENABLE_CLOUDSC_SYCL=ON |
There was a problem hiding this comment.
The second is redundant, it should automatically be switched on when SYCL and Serialbox are enabled due to it being DEFAULT ON
| @@ -70,6 +70,9 @@ if ( HAVE_HIP ) | |||
| find_package(hip REQUIRED) | |||
| endif() | |||
|
|
|||
| ecbuild_add_option( FEATURE SYCL | |||
There was a problem hiding this comment.
This should be linked to a check that SYCL is actually available.
We could do this adding REQUIRED_PACKAGES "IntelSYCL" (see https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2024-0/use-cmake-with-the-compiler.html).
If we want to make this compatible with other SYCL implementations (e.g., whatever OpenSYCL is called these days), we could use this pattern:
find_package(IntelSYCL QUIET)
if(NOT IntelSYCL_FOUND)
find_package(IntelDPCPP QUIET)
endif()
find_package(AdaptiveCpp QUIET)
ecbuild_add_option( FEATURE SYCL
DESCRIPTION "SYCL" DEFAULT OFF
CONDITION IntelSYCL_FOUND OR IntelDPCPP_FOUND OR AdaptiveCpp_FOUND )
src/cloudsc_sycl/CMakeLists.txt
Outdated
| $<INSTALL_INTERFACE:include> | ||
| $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/cloudsc> | ||
| PUBLIC_LIBS | ||
| $<$<BOOL:${LINK_SYCL}>:sycl> |
There was a problem hiding this comment.
Could we use add_sycl_to_target here instead? Seems to be the recommended way in both, Intel and AdaptiveCpp:
https://github.com/AdaptiveCpp/AdaptiveCpp/blob/develop/examples/bruteforce_nbody/CMakeLists.txt
https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2024-0/use-cmake-with-the-compiler.html
| cloudsc_driver(omp_threads, ngptot, nproma); | ||
| } | ||
| else { | ||
| printf("Calling c-cloudsc with the right number of arguments will work better ;-) \n",argc); |
There was a problem hiding this comment.
The compiler kindly pointed this out to me:
/etc/ecmwf/nfs/dh1_home_a/nabr/dwarf-p-cloudsc/nams_sycl/source/cloudsc-dwarf/src/cloudsc_sycl/dwarf_cloudsc.cpp:39:91: warning: data argument not used by format string [-Wformat-extra-args]
printf("Calling c-cloudsc with the right number of arguments will work better ;-) \n",argc);
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
1 warning generated.
Could we change that message to something like the following?
printf("Usage: %s [num_threads ngptot nproma]", argv[0])
There was a problem hiding this comment.
Yes 👍 not the only compiler complaining about this. What do you think about refactoring this for all at a later stage?
| set(CMAKE_CXX_FLAGS "-O3 -L/home/nams/opt/dpcpp/lib -fopenmp -lstdc++") | ||
| set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsycl-early-optimizations -fsycl") | ||
| set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fsycl-targets=nvptx64-nvidia-cuda") | ||
| set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -Xsycl-target-backend --cuda-gpu-arch=sm_80") | ||
| set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -I/usr/local/apps/intel/2021.4.0/compiler/2021.4.0/linux/compiler/include") |
There was a problem hiding this comment.
I think since we're now using add_sycl_to_target, the proper way would be to add the relevant ones to SYCL_FLAGS only. But because we're high-jacking the CMake module from a newer installation, I found it tricky to get to work reliably. So, just as an FYI and not necessary to change anything here.
Co-authored-by: Balthasar Reuter <[email protected]>
Co-authored-by: Balthasar Reuter <[email protected]>
Co-authored-by: Balthasar Reuter <[email protected]>
variants: SCC, SCC-HOIST, SCC-K-CACHING
Tested on NVIDIA A100 via
/cloudsc-bundle build --clean --cmake='CMAKE_CXX_COMPILER="/home/nams/opt/dpcpp/bin/clang++" CMAKE_CXX_FLAGS="-O3 -L/home/nams/opt/dpcpp/lib -fopenmp -fsycl-early-optimizations -fsycl -fsycl-targets=nvptx64-nvidia-cuda -Xcuda-ptxas --maxrregcount=128 -Xsycl-target-backend --cuda-gpu-arch=sm_80 -I/usr/local/apps/intel/2021.4.0/compiler/2021.4.0/linux/compiler/include"' --arch=arch/ecmwf/hpc2020/intel/2021.4.0 --with-gpu --with-sycl --with-serialbox --build-dir=build-syclKtrap=fpin the toolchain fileTested on AMD MI250X (LUMI) via
LLVM ERROR: unsupported libcall legalization