Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against __THREW__ in non-TLS section: .bss #12941

Open
DoDoENT opened this issue Dec 2, 2020 · 30 comments
Labels

Comments

@DoDoENT
Copy link

DoDoENT commented Dec 2, 2020

While building our code with LTO enabled, I get this link error:

wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
<repeats lots of times>

So far I've traced that this happens as soon as I link in libopencv_imgcodecs.a in my app, even though no code ever calls any function from it. My app also uses google test framework. The whole code is compiled with -fno-exceptions, except the libopencv_imgcodecs.a module, which internally uses exceptions.

To work around the issue, I can either not link to libopencv_imgcodecs.a or compile google test framework with -fexceptions.

However, if I add a call to a function from libopencv_imgcodecs.a, linker fails even with LTO disabled with following message:

[build] wasm-ld: error: /Users/dodo/.conan/data/OpenCV/3.2.0.13/nenad/testing/package/e4723bf11185e19bfb3ebb3beb683969bf2bd20f/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss

This happens no matter if exceptions are enabled or disabled in Google Test Framework.

I've googled this error message and found this LLVM pull request by @sbc100 as the one that adds that message to the LLD.

I hope I'll be able to reproduce this using entirely open source components (OpenCV and Google Test Framework), without any proprietary code. If I succeed, I'll share the code here.

I am using emscripten 2.0.9 on MacOS Big Sur.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

A couple quick questions about this issue:

  1. Did you compile all your libraryies and object files from scratch using the same version of emcc (i.e. did you do a clean build).
  2. I'm confused as to how including libopencv_imgcodecs could effect to build if you don't use any object files from it. Are you use -Wl,--whole-archive perhaps.
  3. How are you install emscripten? Are you using emsdk? (i.e. is there a possibility your emscripten and llvm versions are out of sync?).

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

It looks like __THREW__ should be marks as TLS as of #12056. Can you confirm which version of libcompiler-rt (where that symbol lives) is being linked it?

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

When you report an issue like its its often useful to include the full failing a command along with the full output/error of that command. Are you able to do that? I understand that sometimes its not possible for IP reasons.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

Did you compile all your libraryies and object files from scratch using the same version of emcc (i.e. did you do a clean build).

Yes.

I'm confused as to how including libopencv_imgcodecs could effect to build if you don't use any object files from it. Are you use -Wl,--whole-archive perhaps.

No.

How are you install emscripten? Are you using emsdk? (i.e. is there a possibility your emscripten and llvm versions are out of sync?).

I've created a conan package using this recipe

It looks like THREW should be marks as TLS as of #12056. Can you confirm which version of libcompiler-rt (where that symbol lives) is being linked it?

How to do that?

When you report an issue like its its often useful to include the full failing a command along with the full output/error of that command. Are you able to do that? I understand that sometimes its not possible for IP reasons.

I did that.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

If you can see the full error message it should contain within it the full wasm-ld command which contains the full path to compiler-rt bring used.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

When you report an issue like its its often useful to include the full failing a command along with the full output/error of that command. Are you able to do that? I understand that sometimes its not possible for IP reasons.

I did that.

Sorry, that is great and very useful. But I was referring the error message included in the bug report itself. The wasm-ld error line which is part of the full emcc error output. If you could include the full emcc error message it would be useful as it contains (for example) the full wasm-ld command that failed.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

If you can see the full error message it should contain within it the full wasm-ld command which contains the full path to compiler-rt bring used.

This is the full output of ninja:

$ ninja GTestTest
[1/1 3.1/sec] Linking CXX executable bin/GTestTest.js
FAILED: bin/GTestTest.js 
: && /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/em++ -DNDEBUG -O2 -O2   -mmacosx-version-min=10.14 CMakeFiles/GTestTest.dir/GTestTest/Source/GTestTest.cpp.o -o bin/GTestTest.js  -dead_strip  lib/libgtest.a  lib/libgtest_main.a  opencv/lib/libopencv_imgcodecs.a  lib/libgtest.a  opencv/lib/libopencv_imgproc.a  opencv/lib/libopencv_core.a  -lstdc++  opencv/3rdparty/lib/liblibjpeg.a  opencv/3rdparty/lib/liblibpng.a  opencv/3rdparty/lib/libzlib.a  -dead_strip && :
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: opencv/lib/libopencv_imgcodecs.a(loadsave.cpp.o): relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
em++: error: '/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/bin/wasm-ld -o bin/GTestTest.wasm CMakeFiles/GTestTest.dir/GTestTest/Source/GTestTest.cpp.o lib/libgtest.a -L/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/system/local/lib lib/libgtest_main.a -L/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/system/lib opencv/lib/libopencv_imgcodecs.a -L/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm lib/libgtest.a opencv/lib/libopencv_imgproc.a opencv/lib/libopencv_core.a opencv/3rdparty/lib/liblibjpeg.a opencv/3rdparty/lib/liblibpng.a opencv/3rdparty/lib/libzlib.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libc.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libcompiler_rt.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libc++-noexcept.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libc++abi-noexcept.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libdlmalloc.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libpthread_stub.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libc_rt_wasm.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libsockets.a -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined --import-memory --strip-debug --export-table --export main --export stackSave --export stackRestore --export stackAlloc --export __wasm_call_ctors --export __errno_location --export malloc --export free --export __cxa_is_pointer_type --export __cxa_can_catch --export _ZSt18uncaught_exceptionv --export setThrew --export _get_tzname --export _get_daylight --export _get_timezone --export realloc --export testSetjmp --export saveSetjmp -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (1)
ninja: build stopped: subcommand failed.

It appears to use /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm/libcompiler_rt.a, the compiler-rt bundled with emscripten 2.0.9.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

Also the output when LTO is enabled:

FAILED: bin/GTestTest.js 
: && /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/em++ -DNDEBUG -O2 -O2   -mmacosx-version-min=10.14 -flto CMakeFiles/GTestTest.dir/GTestTest/Source/GTestTest.cpp.o -o bin/GTestTest.js  -dead_strip  lib/libgtest.a  lib/libgtest_main.a  opencv/lib/libopencv_imgcodecs.a  lib/libgtest.a  opencv/lib/libopencv_imgproc.a  opencv/lib/libopencv_core.a  -lstdc++  opencv/3rdparty/lib/liblibjpeg.a  opencv/3rdparty/lib/liblibpng.a  opencv/3rdparty/lib/libzlib.a  -dead_strip && :
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__THREW__` in non-TLS section: .bss
wasm-ld: error: lto.tmp: relocation R_WASM_MEMORY_ADDR_TLS_SLEB cannot be used against `__threwValue` in non-TLS section: .bss
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
em++: error: '/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/bin/wasm-ld -o bin/GTestTest.wasm CMakeFiles/GTestTest.dir/GTestTest/Source/GTestTest.cpp.o lib/libgtest.a -L/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/system/local/lib lib/libgtest_main.a -L/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/system/lib opencv/lib/libopencv_imgcodecs.a -L/Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto lib/libgtest.a opencv/lib/libopencv_imgproc.a opencv/lib/libopencv_core.a opencv/3rdparty/lib/liblibjpeg.a opencv/3rdparty/lib/liblibpng.a opencv/3rdparty/lib/libzlib.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libc.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libcompiler_rt.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libc++-noexcept.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libc++abi-noexcept.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libdlmalloc.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libpthread_stub.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libc_rt_wasm.a /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/cache/wasm-lto/libsockets.a -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr --allow-undefined --import-memory --strip-debug --export-table --export main --export stackSave --export stackRestore --export stackAlloc --export __wasm_call_ctors --export __errno_location --export malloc --export free --export _ZSt18uncaught_exceptionv --export __cxa_find_matching_catch --export __cxa_is_pointer_type --export __cxa_can_catch --export setThrew --export _get_tzname --export _get_daylight --export _get_timezone --export realloc --export testSetjmp --export saveSetjmp -z stack-size=5242880 --initial-memory=16777216 --no-entry --max-memory=16777216 --global-base=1024' failed (1)
ninja: build stopped: subcommand failed.

Also, feel free to play with the repository I've prepared for demonstrating the crash.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

It looks like loadsave.cpp.o with threading support enabled (either via -pthread or -s USE_THREADS) but the final binary is not compiled with thread support so that non-threaded compiler-rt is being uses. Is that possible?

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

We are trying to support the linking thread-supporting libraries into non-threaded application, so maybe we can avoid this error message. @tlively, this is the kind of thing we want to allow right?

@DoDoENT DoDoENT closed this as completed Dec 2, 2020
@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

I think this might be something we can/should fix upstream in llvm..

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

It looks like loadsave.cpp.o with threading support enabled (either via -pthread or -s USE_THREADS) but the final binary is not compiled with thread support so that non-threaded compiler-rt is being uses. Is that possible?

I doubt. It's mostly original OpenCV's cmake build script, which correctly enables threads if target platform supports it. Also keep in mind that the very same version of OpenCV with same CMake build script works correctly with emscripten 1.39.16 and fastcomp backend (something we still use in production and are wishing to move away from).

I've built now with ninja -v in order to inspect all compile flags on loadsave.cpp:

[183/212 57.5/sec] /usr/local/bin/ccache /Users/dodo/.conan/data/emsdk_installer/2.0.9/microblink/stable/package/743cf0321be3152777da4d05247a66d1552e70a2/upstream/emscripten/em++ -DHAVE_SWEATER -DMB_NOEXCEPT_EXCEPT_BADALLOC=TNUN_NOEXCEPT_EXCEPT_BADALLOC -DOCV_EXCEPTIONS_DISABLED=1 -DTNUN_MALLOC_OVERCOMMIT=TNUN_OVERCOMMIT_Full -DTNUN_NOEXCEPT_EXCEPT_BADALLOC=noexcept -DTNUN_OVERCOMMIT_Disabled=0 -DTNUN_OVERCOMMIT_Full=2 -DTNUN_OVERCOMMIT_Partial=1 -D_GLIBCXX_USE_CXX11_ABI=1 -D__OPENCV_BUILD=1 -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/3rdparty/libpng -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/3rdparty/libjpeg -Iopencv/3rdparty/zlib -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/3rdparty/zlib -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/modules/imgcodecs/include -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/modules/imgcodecs/src -Iopencv/modules/imgcodecs -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/modules/core/include -I/Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/modules/imgproc/include -isystem . -fPIC   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -pthread -Qunused-arguments -march=i686 -fomit-frame-pointer -ffast-math -mno-sse2 -mno-avx -mno-sse3 -mno-ssse3 -mno-sse4.1 -mno-sse4.2 -ffunction-sections -fvisibility=hidden -fvisibility-inlines-hidden -Wno-deprecated-declarations -DNDEBUG -O2  -DNDEBUG -fvisibility-inlines-hidden -fstrict-aliasing -fstrict-enums -fvisibility=hidden -fvisibility-inlines-hidden -fPIC -std=gnu++2a -mconstant-cfstrings -mmacosx-version-min=10.14 -fomit-frame-pointer -ffunction-sections -fmerge-all-constants -fno-stack-protector -DNDEBUG -fno-unwind-tables -fno-asynchronous-unwind-tables -flto -fno-exceptions -fno-rtti -fno-threadsafe-statics -g -Os -ffast-math -ffp-contract=fast -std=gnu++14 -Wno-error=non-virtual-dtor -fexceptions -std=gnu++14 -MD -MT opencv/modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/src/loadsave.cpp.o -MF opencv/modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/src/loadsave.cpp.o.d -o opencv/modules/imgcodecs/CMakeFiles/opencv_imgcodecs.dir/src/loadsave.cpp.o -c /Users/dodo/Work/CloneOuts/opencv-emscripten-crash/opencv/modules/imgcodecs/src/loadsave.cpp

@DoDoENT DoDoENT reopened this Dec 2, 2020
@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

So, basically, only -march=i686 is suspicious (I need to investigate where this comes from) - but no threading is enabled anywhere.

We still cannot use threading in WASM as mobile browsers don't support it.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

So, basically, only -march=i686 is suspicious (I need to investigate where this comes from) - but no threading is enabled anywhere.

OK, this comes from the original OpenCV's build script:

  if(X86 AND NOT MINGW64 AND NOT X86_64 AND NOT APPLE)
    add_extra_compiler_option(-march=i686)
  endif()

in OpenCVCompilerOptions.cmake. So, I assume that Emscripten still impersonates X86?

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

I've commented-out the above cmake snippet and it didn't help - same error still occurs.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

OK, now I see, OpenCV's build script appends -pthread. This didn't seem to bother fastcomp and emscripten 1.39.16.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

Indeed, and its something we want to support .. I think we can find a way to have the linker accept this kind of thing.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

Anyway, the resulting WASM won't work on Safari and Android Chrome, would it? Those browsers don't support wasm threads anyway, so I definitely don't want any thread-related stuff in my binary. Thank you for the hints and guidelines.

The sample in the repro repository now works (after I remove -pthread flag). I'm now testing on my production codebase to see if it will finally work or if I'll find yet another crash 😛

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

The idea is that the resulting binary will still be effectively single-thread, even though it contains atomic instructions. Thanks to a recent addition such binaries should run on non-thread-supported browser: WebAssembly/threads#144. At least the is the idea. But its certainly safer to avoid compiling with thread support in the first place.

@DoDoENT
Copy link
Author

DoDoENT commented Dec 2, 2020

😂 It appears that removing -pthread from OpenCV somehow magically fixed #12940.

@tlively
Copy link
Member

tlively commented Dec 2, 2020

Yes, we would like to support this without errors if possible. Right now __THREW__ and __threwValue are considered thread local only in compilation units that have atomics and bulk-memory enabled (i.e. compilation units with -pthread). This is a problem in mixed builds because all the compilation units have to agree on whether or not those are thread local. I can think of two solutions.

  1. We change the code to unconditionally make __THREW__ and __threwValue thread local. In compilation units without bulk-memory and atomics enabled, that thread-local designation will later be stripped, which will have the side effect of marking the object file as incompatible with thread-enabled object files. The link will still fail, but it will have failed in an expected way. We actually had this implemented for a bit, but we considered this behavior a bug and changed it: https://reviews.llvm.org/D88323.

  2. We change __THREW__ and __threwValue to be WebAssembly globals so they can always be accessed in a uniform way whether or not pthreads is enabled. This solution would have to wait for the current upstream work on representing WebAssembly tables, globals, and reference types in LLVM IR to finish.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

Couldn't we maybe address this in the linker? The linker already handles TLS data in the data linking case by kind of lowering it away, right? In that case does it matter than R_WASM_MEMORY_ADDR_TLS_SLEB is found against a symbol in a non-TLS section? i.e. since all sections are non-TLS in the static linking case maybe we don't need that error?

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

I guess it would mean merging TLS and regular data and having the internal/fake __tls_base be the same as __memory_base (this would also save a global). I can't remember is we already do this or not.

@tlively
Copy link
Member

tlively commented Dec 2, 2020

In mixed builds, the method of accessing the variable will be different in different compilation units. When -pthread is enabled and the variables are thread-local, accesses are to a global plus an offset. When -pthread is not enabled, accesses do not involve the global. I guess we could solve this in the linker by generating a TLS base of 0 and setting the TLS offsets to be equal to the actual addresses or something like that. Converting TLS to normal data in the linker like that would work when the output does not use a shared memory, but it would not be possible to go the other direction and convert normal data to TLS when the output does use a shared memory. Due to that asymmetry, I don't think this is a problem we should attempt to solve in the linker.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 2, 2020

IIUC we do already effectively convert TLS to normal data in the linker, right? Isn't that exactly what https://reviews.llvm.org/D91115 does?

The limitation is that all the object files have to agree on the TLS-ness of a given symbol. I guess the question is, is it worth trying to remove that restriction? It seems like your are saying one.

@tlively
Copy link
Member

tlively commented Dec 3, 2020

IIUC we do already effectively convert TLS to normal data in the linker, right? Isn't that exactly what https://reviews.llvm.org/D91115 does?

Oh right, I forgot about that change 👍

The limitation is that all the object files have to agree on the TLS-ness of a given symbol. I guess the question is, is it worth trying to remove that restriction? It seems like your are saying one.

My understanding is that it would be possible to remove that restriction only for links without shared memory. I think it would be surprising for users if the type of memory used in the link controlled whether or not compilation units were allowed to disagree about the TLS-ness of a variable.

@sbc100
Copy link
Collaborator

sbc100 commented Dec 3, 2020

Right, I'm only talking about single-theads links (without shared memory).

The win is normal single-threaded users to not having to go though all your sub-projects build systems to try to remove any stray -pthread arguments from the various places where CFLAGS get specified.

But maybe its easier/better to try to improve the error reporting in this case?

@tlively
Copy link
Member

tlively commented Dec 3, 2020

Hmm, that's a good point. The general problem of trying to allow mixed pthread and non-pthread code only works if the result is single-threaded, so this particular problem isn't special in that way. I think I'm convinced that removing this restriction would be useful, then 👍

@stale
Copy link

stale bot commented Apr 17, 2022

This issue has been automatically marked as stale because there has been no activity in the past year. It will be closed automatically if no further activity occurs in the next 30 days. Feel free to re-open at any time if this issue is still relevant.

@stale stale bot added the wontfix label Apr 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants