Most implementations have low tolerance to corrupted database files

[debrouxl]

Hello,

tkrzw_crashes_202404_01.tar.gz

Here's a tarball containing a set of redundant corrupted files which crash at least one of the commands listed below, and the corresponding terminal output (crashes_*.txt files):

tkrzw_dbm_util inspect "$file"
tkrzw_dbm_util rebuild --restore "$file"
tkrzw_dbm_util list --items 4294967296 "$file"
tkrzw_dbm_util set "$file" key0 value0
tkrzw_dbm_util remove "$file" key0

in tkrzw 1.0.27 built thusly under Debian sid amd64:

apt download tkrzw
# As of 2024/04, on Debian sid amd64, this downloads 1.0.27, which is the latest upstream version.
cd tkrzw-...
AFL_USE_ASAN=1 CC=afl-clang-fast CXX=afl-clang-fast++ dpkg-buildpackage -us -uc -b -j4
dpkg -i ... # currently libtkrzw1t64_1.0.27-2_amd64.deb tkrzw-utils_1.0.27-2_amd64.deb

Most of the crashes on those files are controlled asserts caused by attempts to allocate terabytes of memory or more; however, there are also wild pointer accesses, heap-based buffer overflows, etc.
Only the tkh file type didn't fall to afl-fuzz (yet); tkmb tkmc tkmt tks tksh tkst tkt did, most of them for all five commands, often within the first few seconds of fuzzing, if not the first dozens of milliseconds. Some of them reached ~2% crash rate.

FTR:

• this is not just an empty fuzzing exercise - as I described in the e-mail to oss-sec linked below, what got me interested in fuzzing libdb at first, and later other DBM implementations, was DoS consecutive to on-disk DB corruption caused by a forced computer power down. When the contents of a corrupted database can't be salvaged because the appropriate tools can't behave while dealing with such databases (dividing by zero, allocating excessive amounts of memory, performing invalid memory accesses which crash even in non-instrumented builds with segmentation fault or bus errors, entering infinite loops, etc.), as is the case here, there's room for fixing.
• nevertheless, tkrzw is a slight reliability improvement over all of QDBM, Tokyo Cabinet and Kyoto Cabinet, which have a higher variety of memory and logic errors ( https://seclists.org/oss-sec/2018/q2/206 ).

The initial corpus of valid files was built by

mkdir /dev/shm/tkrzw_fuzz
cd /dev/shm/tkrzw_fuzz
for ext in tkh tkt tks tkmt tkmb tkmc tksh tkst; do
	mkdir "input_$ext"
	tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 "input_$ext/empty.$ext"
	tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 "input_$ext/one.$ext"
	tkrzw_dbm_util set "input_$ext/one.$ext" key1 value1
done

for ext in tkh tkt; do
	for crc in 8 16 32; do
		for comp in zlib zstd lz4 lzma rc4 aes; do
			cp "input_$ext/one.$ext" "input_$ext/one_crc${crc}_comp${comp}.$ext"
			tkrzw_dbm_util rebuild --record_crc "$crc" --record_comp "$comp" "input_$ext/one_crc${crc}_comp${comp}.$ext"
		done
	done
done

tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 input_tkh/ten.tkh
tkrzw_dbm_util create --alloc_init 4096 --buckets 1024 input_tkt/ten.tkt
for i in `seq 0 9`; do
	tkrzw_dbm_util set input_tkh/ten.tkh "key$i" "value$i"
	tkrzw_dbm_util set input_tkt/ten.tkt "key$i" "value$i"
done

afl-fuzz invocations:

mkdir /dev/shm/tkrzw_fuzz/tk_tmpdir{m,1,2,3,4,5,6,7}_{wd,afl,tmpdir}

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkh -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkt -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tks -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkmt -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkmb -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkmc -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tksh -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util inspect @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i /dev/shm/tkrzw_fuzz/input_tkst -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util inspect @@

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util rebuild --restore @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util rebuild --restore @@

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util list --items 4294967296 @@
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util list --items 4294967296 @@

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util set @@ key0 value0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util set @@ key0 value0

cd /dev/shm/tkrzw_fuzz/tk_tmpdirm_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdirm_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkh -M tkh_m -Z -e tkh -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir1_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir1_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkt -M tkt_s1 -Z -e tkt -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir2_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir2_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tks -M tks_s2 -Z -e tks -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir3_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir3_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmt -M tkmt_s3 -Z -e tkmt -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir4_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir4_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmb -M tkmb_s4 -Z -e tkmb -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir5_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir5_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkmc -M tkmc_s5 -Z -e tkmc -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir6_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir6_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tksh -M tksh_s6 -Z -e tksh -- tkrzw_dbm_util remove @@ key0
cd /dev/shm/tkrzw_fuzz/tk_tmpdir7_wd
AFL_NO_AFFINITY=1 AFL_TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_afl TMPDIR=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir TMP=/dev/shm/tkrzw_fuzz/tk_tmpdir7_tmpdir afl-fuzz -i- -o /dev/shm/tkrzw_fuzz/output_tkst -M tkst_s7 -Z -e tkst -- tkrzw_dbm_util remove @@ key0

I then massaged the output folders for easier use on the maintainer side :)

[debrouxl]

Ping ? It's been a month :)

[estraier]

Thanks for the report (and sorry for the belated response).

Let me know of the goal of this experiment.
If the process crashes during an operation setting record "A", it's natural that the record "A" is not recovered.
If the process crashes during an operation setting record "B" after setting record "A", it is expected that record "A" is recovered.
And, your code seems to just setting one record and the process crashes during the operation.
Then, what's the expected behavior?

[debrouxl]

The expected behaviour is not to crash, at the very least :)
While it's clear that the attempted operations, especially adding / removing / modifying records, do not have to complete successfully on a broken database, the database system must not allocate excessive amounts of memory (or abort the program through an assert trying to do that), read memory outside the allocated areas (Out Of Bound reads from stack, heap, global variables), write memory outside the allocated areas (Out Of Bounds writes, a.k.a. "memory corruption"), divide by zero, perform improper memory accesses which yield bus errors, etc.
Beyond being user-unfriendly, especially a complete failure to recover a database which was already broken and usually contains data useful to the user (instead of a failure to salvage a subset of the data, when possible), the aforementioned classes of misbehaviour are vulnerabilities. Some of them (and probably all of the ones the fuzzer found in tkrzw so far) are just Denial of Service (DoS), but memory corruption often yields arbitrary code execution / remote code execution, if the corruption primitive is powerful enough and/or can be sufficiently repeated.

For the record:

• among the DBM(-like) implementations, Berkeley DB, GDBM, TDB and MDBx went through more or less significant rounds of bugfixes, in order not to crash upon corrupted databases. For instance, as a result, GDBM could be integrated to OSS-Fuzz and benefit from Google's fuzzing infrastructure. Last time I checked, LMDB had not been fixed, though.
  Well, Berkeley DB still crashes easily, even after slowly, and reactively, fixing dozens of CVE-numbered vulnerabilities, but the latest Berkeley DB version still contains fewer ways to crash than older versions...
  I reported similar fuzzer-generated input crashing various other code bases to their respective maintainers, who fixed the issues.
• in 2017-2018, I attempted to send you e-mails about the worse issues in QDBM, Tokyo Cabinet and Kyoto Cabinet I mentioned above and at https://seclists.org/oss-sec/2018/q2/206 , at the e-mail address listed on the corresponding home pages, but I never received a reply.