Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Build Error with Intel OneAPI Compilers: bytedelta/bytedelta.c #546

Closed
ax3l opened this issue Aug 19, 2023 · 2 comments
Closed

Build Error with Intel OneAPI Compilers: bytedelta/bytedelta.c #546

ax3l opened this issue Aug 19, 2023 · 2 comments

Comments

@ax3l
Copy link
Contributor

ax3l commented Aug 19, 2023 •
edited
Loading

Describe the bug
In Spack, we saw a build issue with the Intel OneAPI compilers.

The issue is in this line:
https://github.com/Blosc/c-blosc2/blob/v2.10.2/plugins/filters/bytedelta/bytedelta.c#L34

...
[ 27%] Building C object blosc/CMakeFiles/blosc2_shared.dir/__/plugins/codecs/zfp/src/decode4d.c.o
cd /tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-build-75yatje/blosc && /builds/spack/spack/lib/spack/env/oneapi/icx -DBLOSC_SHARED_LIBRARY -DUSING_CMAKE -Dblosc2_shared_EXPORTS -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-build-75yatje/blosc -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/include -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/plugins/codecs/zfp/include -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/blosc -I/home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/lz4-1.9.4-mefylb2fjh3bwfcj7b3dqa3astze5aax/include -I/home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/zstd-1.5.5-427ezbxdqpynchbt4vmkjyjh4l6drauc/include -isystem /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/zlib-1.2.13-rmxxggtwwt2lf6qit7o7m3hu4f6utywl/include -std=gnu99 -O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Wextra -pthread -MD -MT blosc/CMakeFiles/blosc2_shared.dir/__/plugins/codecs/zfp/src/decode4d.c.o -MF CMakeFiles/blosc2_shared.dir/__/plugins/codecs/zfp/src/decode4d.c.o.d -o CMakeFiles/blosc2_shared.dir/__/plugins/codecs/zfp/src/decode4d.c.o -c /tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/plugins/codecs/zfp/src/decode4d.c
[ 57%] Building C object blosc/CMakeFiles/blosc2_shared.dir/__/plugins/filters/bytedelta/bytedelta.c.o
cd /tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-build-75yatje/blosc && /builds/spack/spack/lib/spack/env/oneapi/icx -DBLOSC_SHARED_LIBRARY -DUSING_CMAKE -Dblosc2_shared_EXPORTS -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-build-75yatje/blosc -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/include -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/plugins/codecs/zfp/include -I/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/blosc -I/home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/lz4-1.9.4-mefylb2fjh3bwfcj7b3dqa3astze5aax/include -I/home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/zstd-1.5.5-427ezbxdqpynchbt4vmkjyjh4l6drauc/include -isystem /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/zlib-1.2.13-rmxxggtwwt2lf6qit7o7m3hu4f6utywl/include -std=gnu99 -O3 -DNDEBUG -fPIC -fvisibility=hidden -Wall -Wextra -pthread -MD -MT blosc/CMakeFiles/blosc2_shared.dir/__/plugins/filters/bytedelta/bytedelta.c.o -MF CMakeFiles/blosc2_shared.dir/__/plugins/filters/bytedelta/bytedelta.c.o.d -o CMakeFiles/blosc2_shared.dir/__/plugins/filters/bytedelta/bytedelta.c.o -c /tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/plugins/filters/bytedelta/bytedelta.c
/tmp/root/spack-stage/spack-stage-c-blosc2-2.10.1-75yatjedj3aw7dzjtby6faama4dxh33d/spack-src/plugins/filters/bytedelta/bytedelta.c:34:54: error: '__builtin_ia32_palignr128' needs target feature ssse3
bytes16 simd_concat(bytes16 hi, bytes16 lo) { return _mm_alignr_epi8(hi, lo, 15); }
                                                     ^
/opt/intel/oneapi/compiler/2023.2.0/linux/lib/clang/17/include/tmmintrin.h:152:13: note: expanded from macro '_mm_alignr_epi8'
  ((__m128i)__builtin_ia32_palignr128((__v16qi)(__m128i)(a), \
            ^
1 error generated.
make[2]: *** [blosc/CMakeFiles/blosc2_shared.dir/build.make:681: blosc/CMakeFiles/blosc2_shared.dir/__/plugins/filters/bytedelta/bytedelta.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
...

This looks to me like a compiler flag to activate SSE3 might be missing (-msse3?) and/or a test macro to guard this section in the bytedelta.c callsite might need to be added?

To Reproduce
Downloading the OneAPI toolchain and compiling with CMake.

These tools are LLVM based, so one can consider them as Clang-ish.
The CMake compiler identification string is IntelLLVM.

OS: linux-ubuntu20.04-x86_64
CXX: icpx
CC: icx

Note: the classic Intel compilers (icc/icpc; CMake string Intel) are deprecated and will soon be EOL. Intel OneAPI are the new compilers from Intel replacing them.

Expected behavior
Should compile.

Logs

CMake:

$ cmake ... -DDEACTIVATE_LZ4=OFF -DDEACTIVATE_LIZARD=OFF -DDEACTIVATE_SNAPPY=OFF -DDEACTIVATE_ZLIB=OFF -DDEACTIVATE_ZSTD=OFF -DPREFER_EXTERNAL_LIZARD=ON -DPREFER_EXTERNAL_LZ4=ON -DPREFER_EXTERNAL_ZLIB=ON -DPREFER_EXTERNAL_ZSTD=ON -DDEACTIVATE_AVX2=OFF -DBUILD_TESTS:BOOL=OFF -DBUILD_BENCHMARKS:BOOL=OFF -DBUILD_EXAMPLES:BOOL=OFF

-- The C compiler identification is IntelLLVM 2023.2.0
-- The ASM compiler identification is IntelLLVM
-- Found assembler: /builds/spack/spack/lib/spack/env/oneapi/icx
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /builds/spack/spack/lib/spack/env/oneapi/icx - skipped
-- Detecting C compile features
-- Detecting C compile features - done
Configuring for Blosc version: 2.10.1
-- Found LZ4 library: /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/lz4-1.9.4-mefylb2fjh3bwfcj7b3dqa3astze5aax/lib/liblz4.so
-- Could NOT find ZLIB_NG (missing: ZLIB_NG_LIBRARY ZLIB_NG_INCLUDE_DIR) 
-- Found ZLIB: /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/zlib-1.2.13-rmxxggtwwt2lf6qit7o7m3hu4f6utywl/lib/libz.so (found version "1.2.13")  
-- Found ZSTD library: /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/zstd-1.5.5-427ezbxdqpynchbt4vmkjyjh4l6drauc/lib/libzstd.so
-- Building for system processor x86_64
-- Building for compiler ID IntelLLVM
CMake Warning at CMakeLists.txt:294 (message):
  Unable to determine which x86_64 hardware features are supported by the C
  compiler (IntelLLVM 2023.2.0).


-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE  
CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libchkp.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libcilkrts.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libcilkrts.so.5'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libimf.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libintlc.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libintlc.so.5'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libirc.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libpdbx.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libpdbx.so.5'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libsvml.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libirng.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/liboffload.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/liboffload.so.5'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/cilk_db.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libistrconv.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libgfxoffload.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libioffload_host.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist:
  'NOT-FOUND/libioffload_host.so.5'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist:
  'NOT-FOUND/libioffload_target.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist:
  'NOT-FOUND/libioffload_target.so.5'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/libmpx.so'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:733 (message):
  system runtime library file does not exist: 'NOT-FOUND/offload_main'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


CMake Warning at /home/software/spack/[padded-to-256-chars]/linux-ubuntu20.04-x86_64/oneapi-2023.2.0/cmake-3.26.3-fymkmlpq3qpdojzbdwg5hagzbrzvefwz/share/cmake-3.26/Modules/InstallRequiredSystemLibraries.cmake:743 (message):
  system runtime library file does not exist: 'NOT-FOUND/irml'
Call Stack (most recent call first):
  CMakeLists.txt:543 (include)


-- Configuring done (0.6s)
-- Generating done (0.0s)
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING
    DEACTIVATE_LIZARD
    DEACTIVATE_LZ4
    DEACTIVATE_SNAPPY
    PREFER_EXTERNAL_LIZARD

System information:

  • OS: [e.g. OSX]
  • Compiler [e.g. gcc, clang]
  • Version [e.g. 2.0.1]

Additional context
We use C-Blosc2 in ADIOS2.9+ which is used in openPMD-api, which we use in various HPC applications and data science tools.

We build our software stack for HPC systems with Spack.io, and PR spack/spack#39385 introduces C-Blosc2 support (over C-Blosc1) to ADIOS2. Spack runs a lot of HPC compiler tests, which is how this was caught.
@ax3l ax3l mentioned this issue Aug 19, 2023
Merged
@t20100
Copy link
Contributor

t20100 commented Feb 6, 2024 •
edited
Loading

Hi,

I have a similar issue while building bytedelta in hdf5plugin (see silx-kit/hdf5plugin#295 (comment)).
The following C code assumes SSSE3 is available while this depends on the compilation options.

https://github.com/Blosc/C-Blosc2/blob/f0183767d73073eb41bbf97bf56554c7a55c7dbf/plugins/filters/bytedelta/bytedelta.c#L21-L23

Adding a test of the SSSE3 availability as for the other SIMD instruction sets (SSE2, AVX*) elsewhere in the code should prevent this.

@t20100 t20100 mentioned this issue Feb 7, 2024
Merged
@FrancescAlted
Copy link
Member

We have merged #586, so closing. Just reopen if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@FrancescAlted @ax3l @t20100