Add C bindings for custom Sources and Sinks #3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ncopa
pushed a commit
to ncopa/snappy
that referenced
this pull request
Feb 22, 2017
The commit 8bfb028 (Improve zippy decompression speed) introduce a crash when snappy is compiled with _FORTIFY_SOURCE with musl libc. Backtrace reveals that it it comes from using memcpy with overlap, which is undefined behavior. We avoid the undefined behavior due to memcpy with overlap by using a temporary uint64_t. Bactrace from core dump created with `make check`: ```` (gdb) bt #0 memcpy (__n=8, __os=0xb38c8367eea, __od=0xb38c8367eeb) at /usr/include/fortify/string.h:48 google#1 snappy::(anonymous namespace)::UnalignedCopy64 (src=0xb38c8367eea, dst=0xb38c8367eeb) at snappy.cc:92 google#2 0x00006fb4c7c31717 in snappy::(anonymous namespace)::IncrementalCopy ( buf_limit=0xb38c8380ee0 "", op_limit=<optimized out>, op=<optimized out>, src=0xb38c8367eea " .\001") at snappy.cc:178 google#3 snappy::SnappyArrayWriter::AppendFromSelf (len=<optimized out>, offset=<optimized out>, this=<synthetic pointer>) at snappy.cc:1131 google#4 snappy::SnappyDecompressor::DecompressAllTags<snappy::SnappyArrayWriter> ( writer=<synthetic pointer>, this=0x7f1d26737050) at snappy.cc:715 google#5 snappy::InternalUncompressAllTags<snappy::SnappyArrayWriter> ( uncompressed_len=<optimized out>, writer=<synthetic pointer>, decompressor=0x7f1d26737050) at snappy.cc:799 google#6 snappy::InternalUncompress<snappy::SnappyArrayWriter> ( writer=<synthetic pointer>, r=0x7f1d26737000) at snappy.cc:789 google#7 snappy::RawUncompress (compressed=compressed@entry=0x7f1d267370c0, uncompressed=0xb38c8367ee0 " content: .\001") at snappy.cc:1149 google#8 0x00006fb4c7c3194d in snappy::RawUncompress (compressed=<optimized out>, n=<optimized out>, uncompressed=<optimized out>) at snappy.cc:1144 google#9 0x00000b38c5c6261a in snappy::BM_UFlat (iters=99, arg=<optimized out>) at snappy_unittest.cc:1371 google#10 0x00000b38c5c69846 in snappy::Benchmark::Run (this=0xb38c8323c40) at snappy-test.cc:192 google#11 0x00000b38c5c5f3fd in RunSpecifiedBenchmarks () at snappy-test.h:485 google#12 main (argc=1, argv=0x7f1d267374b8) at snappy_unittest.cc:1515 ```` Signed-off-by: Natanael Copa <[email protected]>
|
@googlebot rescan |
Min4sT1r1th
added a commit
to Min4sT1r1th/snappy
that referenced
this pull request
Jul 13, 2023
googletest 1.13.0 requires C++14 at least, otherwise you get build errors like these. please note, that this is a source directory with some patches applied from Arch Linux to use the system-installed googletest library. cd /tmp/snappy-1.1.10 cp -v ../snappy.pc.in . '../snappy.pc.in' -> './snappy.pc.in' patch -p1 < ../cmake_add_pkgconfig.patch patching file CMakeLists.txt Hunk google#1 succeeded at 209 (offset 22 lines). Hunk google#2 succeeded at 423 (offset 22 lines). patch -p1 < ../system_gtest.patch patching file CMakeLists.txt Hunk google#2 succeeded at 320 (offset 28 lines). Hunk google#3 succeeded at 369 (offset 28 lines). patch -p1 < ../reenable_rtti.patch patching file CMakeLists.txt mkdir build-demo cd build-demo CXXFLAGS+=" -DNDEBUG" \ cmake -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=/usr/lib -DBUILD_SHARED_LIBS=no -DSNAPPY_USE_BUNDLED_GTEST=OFF -DSNAPPY_USE_BUNDLED_BENCHMARK_LIB=OFF -- The C compiler identification is GNU 13.1.0 -- The CXX compiler identification is GNU 13.1.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: /usr/bin/gcc - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: /usr/bin/g++ - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Looking for sys/mman.h -- Looking for sys/mman.h - found -- Looking for sys/resource.h -- Looking for sys/resource.h - found -- Looking for sys/time.h -- Looking for sys/time.h - found -- Looking for sys/uio.h -- Looking for sys/uio.h - found -- Looking for unistd.h -- Looking for unistd.h - found -- Looking for windows.h -- Looking for windows.h - not found -- Looking for zlibVersion in z -- Looking for zlibVersion in z - found -- Looking for lzo1x_1_15_compress in lzo2 -- Looking for lzo1x_1_15_compress in lzo2 - found -- Looking for LZ4_compress_default in lz4 -- Looking for LZ4_compress_default in lz4 - found -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX - Failed -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX2 -- Performing Test HAVE_VISUAL_STUDIO_ARCH_AVX2 - Failed -- Performing Test HAVE_CLANG_MAVX -- Performing Test HAVE_CLANG_MAVX - Success -- Performing Test HAVE_CLANG_MBMI2 -- Performing Test HAVE_CLANG_MBMI2 - Success -- Performing Test SNAPPY_HAVE_NO_MISSING_FIELD_INITIALIZERS -- Performing Test SNAPPY_HAVE_NO_MISSING_FIELD_INITIALIZERS - Success -- Performing Test HAVE_BUILTIN_EXPECT -- Performing Test HAVE_BUILTIN_EXPECT - Success -- Performing Test HAVE_BUILTIN_CTZ -- Performing Test HAVE_BUILTIN_CTZ - Success -- Performing Test HAVE_ATTRIBUTE_ALWAYS_INLINE -- Performing Test HAVE_ATTRIBUTE_ALWAYS_INLINE - Failed -- Performing Test SNAPPY_HAVE_SSSE3 -- Performing Test SNAPPY_HAVE_SSSE3 - Failed -- Performing Test SNAPPY_HAVE_X86_CRC32 -- Performing Test SNAPPY_HAVE_X86_CRC32 - Failed -- Performing Test SNAPPY_HAVE_NEON_CRC32 -- Performing Test SNAPPY_HAVE_NEON_CRC32 - Failed -- Performing Test SNAPPY_HAVE_BMI2 -- Performing Test SNAPPY_HAVE_BMI2 - Failed -- Performing Test SNAPPY_HAVE_NEON -- Performing Test SNAPPY_HAVE_NEON - Failed -- Looking for mmap -- Looking for mmap - found -- Looking for sysconf -- Looking for sysconf - found -- Found PkgConfig: /usr/bin/pkg-config (found version "1.9.5") -- Checking for one of the modules 'gtest_main' -- Checking for one of the modules 'benchmark' -- Configuring done (5.7s) -- Generating done (0.0s) -- Build files have been written to: /tmp/snappy-1.1.10/build-demo make -w -j 1 VERBOSE=1 make: Entering directory '/tmp/snappy-1.1.10/build-demo' /usr/bin/cmake -S/tmp/snappy-1.1.10 -B/tmp/snappy-1.1.10/build-demo --check-build-system CMakeFiles/Makefile.cmake 0 /usr/bin/cmake -E cmake_progress_start /tmp/snappy-1.1.10/build-demo/CMakeFiles /tmp/snappy-1.1.10/build-demo//CMakeFiles/progress.marks make -f CMakeFiles/Makefile2 all make[1]: Entering directory '/tmp/snappy-1.1.10/build-demo' make -f CMakeFiles/snappy.dir/build.make CMakeFiles/snappy.dir/depend make[2]: Entering directory '/tmp/snappy-1.1.10/build-demo' cd /tmp/snappy-1.1.10/build-demo && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /tmp/snappy-1.1.10 /tmp/snappy-1.1.10 /tmp/snappy-1.1.10/build-demo /tmp/snappy-1.1.10/build-demo /tmp/snappy-1.1.10/build-demo/CMakeFiles/snappy.dir/DependInfo.cmake --color= make[2]: Leaving directory '/tmp/snappy-1.1.10/build-demo' make -f CMakeFiles/snappy.dir/build.make CMakeFiles/snappy.dir/build make[2]: Entering directory '/tmp/snappy-1.1.10/build-demo' [ 7%] Building CXX object CMakeFiles/snappy.dir/snappy-c.cc.o /usr/bin/g++ -DHAVE_CONFIG_H -I/tmp/snappy-1.1.10/build-demo -I/tmp/snappy-1.1.10 -fPIC -fstack-protector-strong -DNDEBUG -Wall -Wextra -fno-exceptions -std=c++11 -MD -MT CMakeFiles/snappy.dir/snappy-c.cc.o -MF CMakeFiles/snappy.dir/snappy-c.cc.o.d -o CMakeFiles/snappy.dir/snappy-c.cc.o -c /tmp/snappy-1.1.10/snappy-c.cc [ 14%] Building CXX object CMakeFiles/snappy.dir/snappy-sinksource.cc.o /usr/bin/g++ -DHAVE_CONFIG_H -I/tmp/snappy-1.1.10/build-demo -I/tmp/snappy-1.1.10 -fPIC -fstack-protector-strong -DNDEBUG -Wall -Wextra -fno-exceptions -std=c++11 -MD -MT CMakeFiles/snappy.dir/snappy-sinksource.cc.o -MF CMakeFiles/snappy.dir/snappy-sinksource.cc.o.d -o CMakeFiles/snappy.dir/snappy-sinksource.cc.o -c /tmp/snappy-1.1.10/snappy-sinksource.cc ... make[2]: Entering directory '/tmp/snappy-1.1.10/build-demo' [ 64%] Building CXX object CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o /usr/bin/g++ -DHAVE_CONFIG_H -I/tmp/snappy-1.1.10 -I/tmp/snappy-1.1.10/build-demo -fPIC -fstack-protector-strong -DNDEBUG -Wall -Wextra -fno-exceptions -std=c++11 -DGTEST_HAS_PTHREAD=1 -MD -MT CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o -MF CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o.d -o CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o -c /tmp/snappy-1.1.10/snappy_unittest.cc In file included from /usr/include/gtest/gtest-message.h:57, from /usr/include/gtest/gtest-assertion-result.h:46, from /usr/include/gtest/gtest.h:64, from /tmp/snappy-1.1.10/snappy_unittest.cc:39: /usr/include/gtest/internal/gtest-port.h:270:2: error: #error C++ versions less than C++14 are not supported. 270 | #error C++ versions less than C++14 are not supported. | ^~~~~ In file included from /usr/include/gtest/gtest-printers.h:115, from /usr/include/gtest/gtest-matchers.h:48, from /usr/include/gtest/internal/gtest-death-test-internal.h:47, from /usr/include/gtest/gtest-death-test.h:43, from /usr/include/gtest/gtest.h:65: /usr/include/gtest/internal/gtest-internal.h:636:58: error: wrong number of template arguments (0, should be 1) 636 | typedef ::std::map<std::string, CodeLocation, std::less<>> RegisteredTestsMap; | ^ In file included from /usr/include/c++/13.1.0/string:49, from /usr/include/c++/13.1.0/random:42, from /tmp/snappy-1.1.10/snappy_unittest.cc:32: /usr/include/c++/13.1.0/bits/stl_function.h:403:12: note: provided for 'template<class _Tp> struct std::less' 403 | struct less : public binary_function<_Tp, _Tp, bool> | ^~~~ /usr/include/gtest/internal/gtest-internal.h:636:59: error: template argument 3 is invalid 636 | typedef ::std::map<std::string, CodeLocation, std::less<>> RegisteredTestsMap; | ^~ /usr/include/gtest/internal/gtest-internal.h: In member function 'bool testing::internal::TypedTestSuitePState::AddTestName(const char*, int, const char*, const char*)': /usr/include/gtest/internal/gtest-internal.h:613:23: error: request for member 'insert' in '((testing::internal::TypedTestSuitePState*)this)->testing::internal::TypedTestSuitePState::registered_tests_', which is of non-class type 'testing::internal::TypedTestSuitePState::RegisteredTestsMap' {aka 'int'} 613 | registered_tests_.insert( | ^~~~~~ /usr/include/gtest/internal/gtest-internal.h: In member function 'bool testing::internal::TypedTestSuitePState::TestExists(const std::string&) const': /usr/include/gtest/internal/gtest-internal.h:619:30: error: request for member 'count' in '((const testing::internal::TypedTestSuitePState*)this)->testing::internal::TypedTestSuitePState::registered_tests_', which is of non-class type 'const testing::internal::TypedTestSuitePState::RegisteredTestsMap' {aka 'const int'} 619 | return registered_tests_.count(test_name) > 0; | ^~~~~ /usr/include/gtest/internal/gtest-internal.h: In member function 'const testing::internal::CodeLocation& testing::internal::TypedTestSuitePState::GetCodeLocation(const std::string&) const': /usr/include/gtest/internal/gtest-internal.h:623:40: error: qualified-id in declaration before 'it' 623 | RegisteredTestsMap::const_iterator it = registered_tests_.find(test_name); | ^~ /usr/include/gtest/internal/gtest-internal.h:624:5: error: 'it' was not declared in this scope; did you mean 'int'? 624 | GTEST_CHECK_(it != registered_tests_.end()); | ^~~~~~~~~~~~ /usr/include/gtest/internal/gtest-internal.h:624:5: error: request for member 'end' in '((const testing::internal::TypedTestSuitePState*)this)->testing::internal::TypedTestSuitePState::registered_tests_', which is of non-class type 'const testing::internal::TypedTestSuitePState::RegisteredTestsMap' {aka 'const int'} 624 | GTEST_CHECK_(it != registered_tests_.end()); | ^~~~~~~~~~~~ /usr/include/gtest/internal/gtest-internal.h:625:12: error: 'it' was not declared in this scope; did you mean 'int'? 625 | return it->second; | ^~ | int make[2]: *** [CMakeFiles/snappy_unittest.dir/build.make:76: CMakeFiles/snappy_unittest.dir/snappy_unittest.cc.o] Error 1 make[2]: Leaving directory '/tmp/snappy-1.1.10/build-demo' make[1]: *** [CMakeFiles/Makefile2:144: CMakeFiles/snappy_unittest.dir/all] Error 2 make[1]: Leaving directory '/tmp/snappy-1.1.10/build-demo' make: *** [Makefile:146: all] Error 2 make: Leaving directory '/tmp/snappy-1.1.10/build-demo'
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uses structures of function pointers and an opaque userdata pointer to mimic the C++ Source and Sink objects.