KEA: segfault in test_gdal_misc_6

[dbaston]

Expected behavior and actual behavior.

test_gdal_misc_6 may segfault when calling CreateCopy with argument pszFilename="/vsimem/test_truncate/||maxlength=5028||foo"

The failure does not occur if gcore/misc.py is run in isolation. It can be reproduced (for me) with pytest gdrivers/kea.py gcore/misc.py

Stack trace

terminate called after throwing an instance of 'H5::LocationException'

Thread 1 "python3" received signal SIGABRT, Aborted.
__pthread_kill_implementation (no_tid=0, signo=6, threadid=140737352425472) at ./nptl/pthread_kill.c:44
44	./nptl/pthread_kill.c: No such file or directory.
(gdb) bt
#0  __pthread_kill_implementation (no_tid=0, signo=6, threadid=140737352425472) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (signo=6, threadid=140737352425472) at ./nptl/pthread_kill.c:78
#2  __GI___pthread_kill (threadid=140737352425472, signo=signo@entry=6) at ./nptl/pthread_kill.c:89
#3  0x00007ffff7c42476 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7c287f3 in __GI_abort () at ./stdlib/abort.c:79
#5  0x00007ffff22a2b9e in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff22ae20c in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
#7  0x00007ffff22ae277 in std::terminate() () from /lib/x86_64-linux-gnu/libstdc++.so.6
#8  0x00007ffff22ae4d8 in __cxa_throw () from /lib/x86_64-linux-gnu/libstdc++.so.6
#9  0x00007ffff01f491e in ?? () from /lib/x86_64-linux-gnu/libhdf5_serial_cpp.so.103
#10 0x00007ffff0275769 in kealib::KEAImageIO::createKEAImage(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, kealib::KEADataType, unsigned int, unsigned int, unsigned int, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >*, kealib::KEAImageSpatialInfo*, unsigned int, unsigned int, int, unsigned long long, unsigned long long, double, unsigned long long, unsigned long long, unsigned int) () from /lib/libkea.so.1.4
#11 0x00007ffff463d6af in KEADataset::CreateLL (pszFilename=0x7fff6c1c7f80 "/vsimem/test_truncate/||maxlength=5028||foo", nXSize=10, nYSize=10, 
    nBandsIn=1, eType=GDT_Byte, papszParamList=0x0) at /home/dan/dev/gdal/frmts/kea/keadataset.cpp:293
#12 0x00007ffff463dab1 in KEADataset::CreateCopy (pszFilename=0x7fff6c1c7f80 "/vsimem/test_truncate/||maxlength=5028||foo", pSrcDs=0x7fff6c005c50, 
    bStrict=1, papszParamList=0x0, pfnProgress=0x7ffff39f5e72 <GDALDummyProgress(double, char const*, void*)>, pProgressData=0x7fff6d25af20)
    at /home/dan/dev/gdal/frmts/kea/keadataset.cpp:364
#13 0x00007ffff5031ea7 in GDALDriver::CreateCopy (this=0x5555561d01b0, pszFilename=0x7fff6c1c7f80 "/vsimem/test_truncate/||maxlength=5028||foo", 
    poSrcDS=0x7fff6c005c50, bStrict=1, papszOptions=0x0, pfnProgress=0x7ffff39f5e72 <GDALDummyProgress(double, char const*, void*)>, 
    pProgressData=0x7fff6d25af20) at /home/dan/dev/gdal/gcore/gdaldriver.cpp:1347
#14 0x00007ffff50321bb in GDALCreateCopy (hDriver=0x5555561d01b0, pszFilename=0x7fff6c1c7f80 "/vsimem/test_truncate/||maxlength=5028||foo", 
    hSrcDS=0x7fff6c005c50, bStrict=1, papszOptions=0x0, pfnProgress=0x0, pProgressData=0x7fff6d25af20) at /home/dan/dev/gdal/gcore/gdaldriver.cpp:1394
#15 0x00007ffff54d7d58 in GDALDriverShadow_CreateCopy (callback_data=0x7fff6d25af20, callback=0x0, options=0x0, strict=1, src=0x7fff6c005c50, 
    utf8_path=0x7fff6c1c7f80 "/vsimem/test_truncate/||maxlength=5028||foo", self=0x5555561d01b0) at extensions/gdal_wrap.cpp:4564
#16 _wrap_Driver_CreateCopy (args=<optimized out>, kwargs=<optimized out>) at extensions/gdal_wrap.cpp:16680

Adding the following catch block prevents the segfault in CreateCopy

    catch (...) {
        CPLError(CE_Failure, CPLE_OpenFailed,
                 "Attempt to create file `%s' failed. Error: Unknown\n", pszFilename);
        return nullptr;
    }

but produces a new one at program exit:

#0  0x00007ffff04ef274 in H5F__close_cb () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#1  0x00007ffff0559253 in ?? () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#2  0x00007ffff06153da in H5SL_try_free_safe () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#3  0x00007ffff055dc91 in H5I_clear_type () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#4  0x00007ffff04e6c70 in H5F_term_package () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#5  0x00007ffff0444f84 in H5_term_library () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#6  0x00007ffff0445a81 in H5close () from /lib/x86_64-linux-gnu/libhdf5_serial.so.103
#7  0x00007ffff020078d in H5::H5Library::termH5cpp() () from /lib/x86_64-linux-gnu/libhdf5_serial_cpp.so.103
#8  0x00007ffff7c45495 in __run_exit_handlers (status=0, listp=0x7ffff7e19838 <__exit_funcs>, run_list_atexit=run_list_atexit@entry=true, 
    run_dtors=run_dtors@entry=true) at ./stdlib/exit.c:113

[rouault]

@gillins are you interested in looking into that ?

[gillins]

Yes I am - will let you know what I find. Thanks for letting me know.

[gillins]

@dbaston have created a PR that fixes this for me in #8786. Does this solve it for you also?

[dbaston]

With these changes I get a segfault at program exit (trace is in the description above). I also get a lot of these messages:

ERROR 6: Maximum file size reached
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!
ERROR 6: Maximum file size reached!

[rouault]

#8788 fixes/works around things