Ubuntu16.04 complile "proxygen"(master version) get DSO error

[loveleon]

compile environment information: ubuntu16.04 64bit gcc-5.4
when i run proxygen deps.sh to compile proxygen.
i got an error:
libtool: link: ranlib .libs/libqcram.a
libtool: link: rm -fr .libs/libqcram.lax
libtool: link: ( cd ".libs" && rm -f "libqcram.la" && ln -s "../libqcram.la" "libqcram.la" )
/bin/bash ../../libtool --tag=CXX --mode=link g++ -std=gnu++14 -pthread -DLIBMC_FBTRACE_DISABLE -Wno-missing-field-initializers -Wno-deprecated -O3 -g -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o CompressionSimulator codec/compress/test/HTTPArchive.o codec/compress/experimental/simulator/Main.o codec/compress/experimental/simulator/CompressionSimulator.o libqpack.la libqcram.la libproxygenhttp.la ../services/libproxygenservices.la -liberty -lwangle -lfolly -lz -lssl -lcrypto -levent -lgflags -lglog -L/usr/lib64 -lpthread -pthread -lfolly -lglog -ldouble-conversion -lboost_system -lboost_thread
libtool: link: g++ -std=gnu++14 -pthread -DLIBMC_FBTRACE_DISABLE -Wno-missing-field-initializers -Wno-deprecated -O3 -g -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o CompressionSimulator codec/compress/test/HTTPArchive.o codec/compress/experimental/simulator/Main.o codec/compress/experimental/simulator/CompressionSimulator.o -pthread ./.libs/libqpack.a -L/usr/lib64 ./.libs/libqcram.a ./.libs/libproxygenhttp.a ../services/.libs/libproxygenservices.a -liberty -lwangle -lz -lssl -lcrypto -levent -lgflags -lpthread -lfolly -lglog -ldouble-conversion -lboost_system -lboost_thread -pthread
/usr/bin/ld: //usr/local/lib/libfolly.a(CacheLocality.cpp.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5'
//lib/x86_64-linux-gnu/libdl.so.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:916: recipe for target 'CompressionSimulator' failed
make[5]: *** [CompressionSimulator] Error 1
make[5]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib/http'
Makefile:1144: recipe for target 'all-recursive' failed
make[4]: *** [all-recursive] Error 1
make[4]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib/http'
Makefile:661: recipe for target 'all' failed
make[3]: *** [all] Error 2
make[3]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib/http'
Makefile:512: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen/lib'
Makefile:418: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/cxh/CLionProjects/proxygen/proxygen'
Makefile:350: recipe for target 'all' failed
make: *** [all] Error 2

i decide to add -ldl LDFLAGS in somewhere CMakeLists.txt to deal with the issues.
But i couldn't known how to add it .
So everyone can help me,give some ideas,thanks very much.

[AO-StreetArt]

I received the same error, I ran the below command to get -ldl into the build command and ensure that the linker could find both libdl.so and libdl.so.2:

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/lib32/:/usr/lib/x86_64-linux-gnu/
export LIBRARY_PATH=$LIBRARY_PATH:/lib32/:/usr/lib/x86_64-linux-gnu/
export LIBS=-ldl

Then re-executed. The error message changes slightly:

libtool: link: g++ -std=gnu++14 -pthread -DLIBMC_FBTRACE_DISABLE -Wno-missing-field-initializers -Wno-deprecated -O3 -g -W -Wall -Wextra -Wno-unused-parameter -fno-strict-aliasing -g -O2 -o CompressionSimulator codec/compress/test/HTTPArchive.o codec/compress/experimental/simulator/Main.o codec/compress/experimental/simulator/CompressionSimulator.o -pthread  ./.libs/libqpack.a -L/usr/lib64 ./.libs/libqcram.a ./.libs/libproxygenhttp.a ../services/.libs/libproxygenservices.a -lwangle -lz -lssl -lcrypto -levent -lgflags -ldl -lpthread -lfolly -lglog -ldouble-conversion -lboost_system -lboost_thread -pthread
/usr/bin/ld: //usr/local/lib/libfolly.a(CacheLocality.cpp.o): undefined reference to symbol 'dlclose@@GLIBC_2.2.5'
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libdl.so: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:916: recipe for target 'CompressionSimulator' failed
make[5]: *** [CompressionSimulator] Error 1
make[5]: Leaving directory '/home/abarry/Documents/cpp/proxygen/proxygen/lib/http'
Makefile:1144: recipe for target 'all-recursive' failed
make[4]: *** [all-recursive] Error 1
make[4]: Leaving directory '/home/abarry/Documents/cpp/proxygen/proxygen/lib/http'
Makefile:661: recipe for target 'all' failed
make[3]: *** [all] Error 2
make[3]: Leaving directory '/home/abarry/Documents/cpp/proxygen/proxygen/lib/http'
Makefile:512: recipe for target 'all-recursive' failed
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory '/home/abarry/Documents/cpp/proxygen/proxygen/lib'
Makefile:418: recipe for target 'all-recursive' failed
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory '/home/abarry/Documents/cpp/proxygen/proxygen'
Makefile:350: recipe for target 'all' failed
make: *** [all] Error 2

The linker is definately having trouble finding the libdl binaries, I'm not quite sure how to get it to pick them up correctly.

[jmccl]

Getting the same error on debian.

[jmccl]

I took a look at this and the source of the problem seems to be that folly, at some point, stopped creating shared libraries.

(This causes the error above because CompressionSimulator links to the folly static library and therefore doesn’t pick up the ‘dl’ dependency. There’s also a boost_context dependency that needs to be added, but even if you get through this, you get a bunch of fPIC errors elsewhere.)

If you force folly to create both shared and static libraries (which it seemed to do by default in the past) and install them, then deps.sh will run clean. However, the folly CMakeLists.txt file reads in part:

option(BUILD_SHARED_LIBS
"If enabled, build folly as a shared library.
This is generally discouraged, since folly does not commit to having
a stable ABI."
OFF
)

I assume the travis-ci build is succeeding because it has some way of getting folly into the ‘correct’ state. However, the open source build – as far as I can tell – broken right now.

I’d appreciate it if someone from FB can chime in.

[Orvid]

@simpkins This sounds like an unintended side effect of the switch over to CMake for Folly and Wangle upstream :)

[loveleon]

@jmccl i ask for help from folly already. You can find the answer from the closed issue.

[petabytestorage]

not sure if this is still unclear, but in case it helps, i use this option to build

libfolly

shared

cmake -DBUILD_SHARED_LIBS=ON

so i can build proxygen with ansible in Ubuntu 18.04 . i found it is important to have -fPIC versions of the dependency libs.

[Rodbourn]

Could you elaborate on getting a clean build with Ubuntu 18.04?

[emgre]

To get a clean build, you need to compile folly and wangle as shared libraries. To do this, modify

proxygen/proxygen/deps.sh

Line 111 in 6903e7b

cmake configure ..

and

proxygen/proxygen/deps.sh

Line 133 in 6903e7b

cmake configure ../wangle

to add -DBUILD_SHARED_LIBS=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON

Here's a sed to do it: sed -i 's/cmake configure \.\./cmake configure -DBUILD_SHARED_LIBS=ON -DCMAKE_POSITION_INDEPENDENT_CODE=ON \.\./' proxygen/deps.sh

I haven't tested this on Ubuntu 18.04, I'm on 16.04, but I guess it will work.

This issue should really be addressed and the CI should be updated to pull a clean version of folly to validate that the issue is fixed.

By the way, the issue @loveleon is refering to is this one: facebook/folly#825

[Rodbourn]

Thank you very much. I had -DBUILD_SHARED_LIBS=ON on line 111, but was missing it from 133, and I was missing -DCMAKE_POSITION_INDEPENDENT_CODE=ON on both.

I got a clean build on 18.04.

It would be nice if they addressed this...

[afrind]

Would one of you like to submit a PR?

[Rodbourn]

I'd be happy to :)

[simpkins]

Sorry, I just had a chance to look at this. I don't think that building folly and wangle as shared libraries is really the ideal fix here. Neither folly nor wangle have stable ABIs, and we never update their soname when binary-incompatible changes are made, so using them as shared libraries makes it very easy to accidentally cause problems for yourself unless you are very careful about managing the library installations. Unless you make sure to always rebuild and reinstall the folly, wangle, and proxygen libraries together at the same time it is easy to end up with incompatible versions of the libraries installed, and nothing will complain except that the code will probably crash at runtime.

I think a better fix is to simply update the proxygen makefiles to simply add "-ldl" to their link lines.
(This would happen automatically if proxygen were built with CMake and used the CMake project config files that folly and wangle install.)

[emgre]

If you want to link statically, you will need more than just dl. Here's the list that I found to work:
-ldl -levent_core -lssl -lboost_context -lboost_regex -lboost_filesystem.

Here's a sed to fix the build:

sed -i 's/\(LIBS="$LIBS $BOOST.*\)"/\1 -ldl -levent_core -lssl"/' proxygen/configure.ac
sed -i 's/\(LIBS="$LIBS -ldouble.*\)"/\1 -lboost_context -lboost_regex -lboost_filesystem"/' proxygen/configure.ac

I agree with you that it would simplify a lot of the building issues if proxygen was using CMake like folly and wangle...

[Rodbourn]

I think a better fix is to simply update the proxygen makefiles to simply add "-ldl" to their link lines.

I didn't find it that simple, I'm trying what @emgre suggests now. With the above in mind, I agree against shared libraries.

[jmccl]

I put in a pull request (#233) to build folly and wangle statically.

[udippant]

This issue should be be resolved after adding cmake build for both folly and Proxygen. Closing.