ZFP+ADIOS2 Leaks memory

[NAThompson]

Describe the bug

It appears that all memory sent into ADIOS and passed through ZFP is leaked.

To Reproduce

• Clone the latest version of ZFP (08adb27dbc7ed91373ba76312aa7e48504a02190), build with -fsanitize=address -fsanitize=undefined to demonstrate that the leaks are not in ZFP.
• Build ADIOS2 on top of the latest commit (currently 1e9c124) with the options:

ADIOS2/build$ cmake -DCMAKE_CXX_COMPILER="g++-9" -DCMAKE_C_COMPILER="gcc-9" -DCMAKE_CXX_FLAGS="-fsanitize=address -fsanitize=undefined" -DADIOS2_USE_MPI=OFF -DCMAKE_BUILD_TYPE=Debug ../
ADIOS2/build$ make -j`nproc`
ADIOS2/build$ ctest -V
278: Test command: /home/4nt/ADIOS2/build/bin/Test.Engine.BP.WriteReadZfpConfig "--gtest_filter=*/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/*" "BP4"
278: Test timeout computed to be: 1500
278: Note: Google Test filter = */BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/*
278: [==========] Running 9 tests from 1 test case.
278: [----------] Global test environment set-up.
278: [----------] 9 tests from ZfpConfigFile/BPWriteReadZfpConfig
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/0
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/0 (11 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/1
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/1 (3 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/2
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/2 (4 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/3
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/3 (3 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/4
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/4 (3 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/5
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/5 (4 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/6
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/6 (6 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/7
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/7 (5 ms)
278: [ RUN      ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/8
278: [       OK ] ZfpConfigFile/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/8 (5 ms)
278: [----------] 9 tests from ZfpConfigFile/BPWriteReadZfpConfig (44 ms total)
278: 
278: [----------] Global test environment tear-down
278: [==========] 9 tests from 1 test case ran. (44 ms total)
278: [  PASSED  ] 9 tests.
278: 
278: =================================================================
278: ==21184==ERROR: LeakSanitizer: detected memory leaks
278: 
278: Direct leak of 360 byte(s) in 9 object(s) allocated from:
278:     #0 0x7fa2d0960f78 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10bf78)
278:     #1 0x7fa2c0e10be9 in stream_open /home/4nt/zfp/src/inline/bitstream.c:428
278:     #2 0x7fa2c9a8c1d8 in adios2::core::compress::CompressZFP::Compress(void const*, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&) const /home/4nt/ADIOS2/source/adios2/operator/compress/CompressZFP.cpp:49
278:     #3 0x7fa2c97d1331 in void adios2::format::BPOperation::SetDataDefault<double>(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&, adios2::core::Variable<double>::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/BPOperation.tcc:34
278:     #4 0x7fa2c97e048c in adios2::format::BPZFP::SetData(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&, adios2::core::VariableBase::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/compress/BPZFP.cpp:53
278:     #5 0x7fa2c8d46384 in void adios2::format::BPSerializer::PutOperationPayloadInBuffer<double>(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/BPSerializer.tcc:375
278:     #6 0x7fa2c93d464d in void adios2::format::BP4Serializer::PutVariablePayload<double>(adios2::core::Variable<double> const&, adios2::core::Variable<double>::Info const&, bool, adios2::core::Variable<double>::Span*) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bp4/BP4Serializer.tcc:120
278:     #7 0x7fa2c8b61cfb in void adios2::core::engine::BP4Writer::PutSyncCommon<double>(adios2::core::Variable<double>&, adios2::core::Variable<double>::Info const&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:99
278:     #8 0x7fa2c8b1d0fc in void adios2::core::engine::BP4Writer::PerformPutCommon<double>(adios2::core::Variable<double>&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:137
278:     #9 0x7fa2c8af4099 in adios2::core::engine::BP4Writer::PerformPuts() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:83
278:     #10 0x7fa2c8af578b in adios2::core::engine::BP4Writer::EndStep() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:94
278:     #11 0x7fa2c9c4be82 in adios2::Engine::EndStep() /home/4nt/ADIOS2/bindings/CXX11/adios2/cxx11/Engine.cpp:122
278:     #12 0x56554f102202 in ZfpRate2DSmallSel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:795
278:     #13 0x56554f1090a6 in BPWriteReadZfpConfig_ADIOS2BPWriteReadZfp2DSmallSel_Test::TestBody() /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:883
278:     #14 0x56554f21b1ed in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278:     #15 0x56554f1fee52 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278:     #16 0x56554f1672ad in testing::Test::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2485
278:     #17 0x56554f16a857 in testing::TestInfo::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2661
278:     #18 0x56554f16d880 in testing::TestCase::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2779
278:     #19 0x56554f1a1db9 in testing::internal::UnitTestImpl::RunAllTests() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:5036
278:     #20 0x56554f221395 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278:     #21 0x56554f204008 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278:     #22 0x56554f196193 in testing::UnitTest::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:4652
278:     #23 0x56554f10cc81 in RUN_ALL_TESTS() (/home/4nt/ADIOS2/build/bin/Test.Engine.BP.WriteReadZfpConfig+0x23fc81)
278:     #24 0x56554f10951b in main /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:902
278:     #25 0x7fa2c1a29b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
278: 
278: Direct leak of 360 byte(s) in 9 object(s) allocated from:
278:     #0 0x7fa2d0960f78 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10bf78)
278:     #1 0x7fa2c0e10be9 in stream_open /home/4nt/zfp/src/inline/bitstream.c:428
278:     #2 0x7fa2c9a8c1d8 in adios2::core::compress::CompressZFP::Compress(void const*, std::vector<unsigned long, std::allocator<unsigned long> > const&, unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, void*, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > > const&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > >&) const /home/4nt/ADIOS2/source/adios2/operator/compress/CompressZFP.cpp:49
278:     #3 0x7fa2c97cefa5 in void adios2::format::BPOperation::SetDataDefault<float>(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&, adios2::core::Variable<float>::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/BPOperation.tcc:34
278:     #4 0x7fa2c97e0004 in adios2::format::BPZFP::SetData(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&, adios2::core::VariableBase::Operation const&, adios2::format::BufferSTL&) const /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bpOperation/compress/BPZFP.cpp:53
278:     #5 0x7fa2c8d439a6 in void adios2::format::BPSerializer::PutOperationPayloadInBuffer<float>(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/BPSerializer.tcc:375
278:     #6 0x7fa2c93ce9b9 in void adios2::format::BP4Serializer::PutVariablePayload<float>(adios2::core::Variable<float> const&, adios2::core::Variable<float>::Info const&, bool, adios2::core::Variable<float>::Span*) /home/4nt/ADIOS2/source/adios2/toolkit/format/bp/bp4/BP4Serializer.tcc:120
278:     #7 0x7fa2c8b5e4c7 in void adios2::core::engine::BP4Writer::PutSyncCommon<float>(adios2::core::Variable<float>&, adios2::core::Variable<float>::Info const&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:99
278:     #8 0x7fa2c8b1c68c in void adios2::core::engine::BP4Writer::PerformPutCommon<float>(adios2::core::Variable<float>&) /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.tcc:137
278:     #9 0x7fa2c8af3c9e in adios2::core::engine::BP4Writer::PerformPuts() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:83
278:     #10 0x7fa2c8af578b in adios2::core::engine::BP4Writer::EndStep() /home/4nt/ADIOS2/source/adios2/engine/bp4/BP4Writer.cpp:94
278:     #11 0x7fa2c9c4be82 in adios2::Engine::EndStep() /home/4nt/ADIOS2/bindings/CXX11/adios2/cxx11/Engine.cpp:122
278:     #12 0x56554f102202 in ZfpRate2DSmallSel(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >) /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:795
278:     #13 0x56554f1090a6 in BPWriteReadZfpConfig_ADIOS2BPWriteReadZfp2DSmallSel_Test::TestBody() /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:883
278:     #14 0x56554f21b1ed in void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278:     #15 0x56554f1fee52 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278:     #16 0x56554f1672ad in testing::Test::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2485
278:     #17 0x56554f16a857 in testing::TestInfo::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2661
278:     #18 0x56554f16d880 in testing::TestCase::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2779
278:     #19 0x56554f1a1db9 in testing::internal::UnitTestImpl::RunAllTests() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:5036
278:     #20 0x56554f221395 in bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2411
278:     #21 0x56554f204008 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:2447
278:     #22 0x56554f196193 in testing::UnitTest::Run() /home/4nt/ADIOS2/thirdparty/GTest/googletest/googletest/src/gtest.cc:4652
278:     #23 0x56554f10cc81 in RUN_ALL_TESTS() (/home/4nt/ADIOS2/build/bin/Test.Engine.BP.WriteReadZfpConfig+0x23fc81)
278:     #24 0x56554f10951b in main /home/4nt/ADIOS2/testing/adios2/engine/bp/operations/TestBPWriteReadZfpConfig.cpp:902
278:     #25 0x7fa2c1a29b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)
278: 
278: SUMMARY: AddressSanitizer: 720 byte(s) leaked in 18 allocation(s).
42/83 Test #278: Engine.BP.*/BPWriteReadZfpConfig.ADIOS2BPWriteReadZfp2DSmallSel/*.BP4 .........***Failed    0.28 sec

• OS/Platform: Ubuntu 18.04
• Build: gcc 9

[williamfgc]

@NAThompson good catch, is this only applying to the small tests? I had trouble with those in the past. Are other zfp tests leaking?

[NAThompson]

All ZFP unit tests are leaking.

I have tracked one down a little bit:

The data returned by FFS_ZFPCompress is leaked; namely the point malloc'd at ffs_zfp.c line 150 is never freed. It is natural to look to the calling function to see if that is responsible; this is ffs_marshall.c, line 2063:

#ifdef ADIOS2_HAVE_ZFP
            /* this should never be true if ZFP is not available */
            size_t ByteCount;
            char *Output = FFS_ZFPCompress(Stream, Rec->DimCount, Rec->Type,
                                           (void *)Data, Count, &ByteCount);
            DataEntry->ElemCount = ByteCount;
            DataEntry->Array = Output;
#endif

I tried to free Output, but the unit test went from leaking to segfaulting. This is where the trail goes cold for me. The DataEntry variable is a local; it feels like I should be able to delete it entirely with no side effects. But something nonlocal is happening here, and I can't follow it.

[NAThompson]

Fixed in 1977.

[chuckatkins]

Let's leave this open until the fix is actually merged.