Test clean-out from #13165 #13259

nabijaczleweli · 2022-03-27T15:48:55Z

Motivation and Context

Description

#13215, #13250, C test helper parallelisation, de-eyesorification of the bits I could think of, removal of unused bits, &c.; see commit messages

How Has This Been Tested?

Hopelessly in a loop for the best part of the past month. Probably approaching 700CPU·h

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Performance enhancement (non-breaking change which improves efficiency)
Code cleanup (non-breaking change which makes code smaller or more readable)
Breaking change (fix or feature that would cause existing functionality to change)
Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
Documentation (a change to man pages or other documentation)

Checklist:

My code follows the OpenZFS code style requirements.
I have updated the documentation accordingly.
I have read the contributing document.
I have added tests to cover my changes. – well, I fixed them, but haven't added any new ones
I have run the ZFS Test Suite with this change applied.
All commit messages are properly formatted and contain Signed-off-by.

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Previously, they'd all be skipped on FreeBSD where share is showmount Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

This thoroughly destroys logapi and races to the log files horribly Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Ref: openzfs#13215 Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

This only worked by accident on FreeBSD, where sum doesn't take flags POSIX guarantees us cksum (Ethernet CRC) ‒ use that instead Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

The users spew udevadm ENOENTs on FreeBSD Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Also: actually accept all the flags in write_d_a Signed-off-by: Ahelenia Ziemiańska <[email protected]>

This confers an >10x speedup on t/z-t/cmd builds (12s -> 1.1s), gets rid of 23 redundant identical automake specs and gitignores, and groups the binaries with their common headers Signed-off-by: Ahelenia Ziemiańska <[email protected]>

This also fixes line welding in test error output Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Fixes rsend_012_pos: 00:04:59.68 SUCCESS: cmp_ds_prop testpool/testfs/vol testpool2/testfs/vol nosource 00:04:59.70 4c4 00:04:59.70 < acltype posix inherited from testpool 00:04:59.70 --- 00:04:59.70 > acltype posix inherited from testpool2 00:04:59.70 11,12c11,12 00:04:59.70 < devices off inherited from testpool 00:04:59.70 < exec on inherited from testpool 00:04:59.70 --- 00:04:59.70 > devices off inherited from testpool2 00:04:59.70 > exec on inherited from testpool2 00:04:59.70 17c17 00:04:59.70 < setuid off inherited from testpool 00:04:59.70 --- 00:04:59.70 > setuid off inherited from testpool2 00:04:59.70 ERROR: cmp_ds_prop testpool@final testpool2@final exited 1 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13250 Closes openzfs#13259

Original error: 23:47:40.59 SUCCESS: eval zfs receive -dFv testpool2 < /mnt/testroot/backdir-rsend/pool-final-p 23:47:40.61 1,23d0 23:47:40.61 < type filesystem - 23:47:40.61 < origin POOL@psnap - 23:47:40.61 < volblocksize - - 23:47:40.61 < acltype nfsv4 inherited from POOL 23:47:40.61 < dnodesize legacy inherited from POOL 23:47:40.61 < atime off local 23:47:40.61 < canmount off local 23:47:40.61 < checksum off local 23:47:40.61 < compression off local 23:47:40.61 < copies 3 local 23:47:40.61 < devices off local 23:47:40.61 < exec off local 23:47:40.61 < quota none default 23:47:40.61 < readonly on local 23:47:40.61 < recordsize 128K local 23:47:40.61 < reservation none default 23:47:40.61 < setuid off local 23:47:40.61 < snapdir hidden local 23:47:40.61 < version 5 - 23:47:40.61 < volsize - - 23:47:40.61 < xattr off local 23:47:40.61 < mountpoint /PREFIX inherited from POOL 23:47:40.61 < jailed on local 23:47:40.62 cannot open 'testpool2/pclone': dataset does not exist 23:47:40.62 ERROR: cmp_ds_prop testpool/pclone testpool2/pclone exited 1 So: (a) actually send all the datasets in -p mode and (b) drop origin for clones sent with -p: 00:38:05.46 SUCCESS: eval zfs receive -dFv testpool2 < /mnt/testroot/backdir-rsend/pool-final-p 00:38:05.48 2c2 00:38:05.48 < origin POOL@psnap 00:38:05.48 --- 00:38:05.48 > origin POOL 00:38:05.49 ERROR: cmp_ds_prop testpool/pclone testpool2/pclone nosource exited 1 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13250 Closes openzfs#13259

Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

Otherwise, they leak past the tests and contaminate the running system, breaking coredumps entirely Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Co-authored-by: Yannick Le Pennec <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

This also fixes zfs_unshare_006_pos, which exposed this Fixes: 2f71caf ("Allow zfs unshare <protocol> -a") Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

This is a valid configuration and both (a) skips the tests if it's unbuilt/not installed and (b) makes it work even if installed outside the system directory (like in /u/l/l/s instead of /l/s) Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

…me each time Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

As found by git -C tests/ grep ^function | grep -vFe '.lua:' -e '.zcp:' | while IFS=":$IFS" read -r _ _ fn _; do [ $(git -C tests/ grep -wF $fn | head -2 | wc -l) -eq 1 ] && echo $fn; done after all rounds this comes out to, sorted: check_slog_state chgusr_exec cksum_files cleanup_pools compare_modes count_ACE dataset_set_defaultproperties ds_is_snapshot get_ACE get_group get_min get_mode get_owner get_rand_checksum get_rand_checksum_any get_rand_large_recsize get_rand_recsize get_user_group getitem indirect_vdev_mapping_size is_dilos log_noresult log_notinuse log_other log_timed_out log_uninitiated log_warning num_jobs_by_cpu plus_sign_check_l plus_sign_check_v record_cksum rwx_node seconds_mmp_waits_for_activity set_cur_usr setup_mirrors setup_raidzs showshares_smb zfs_zones_setup This, of course, doesn't catch recursive ones, or ones that log with their own function name as a prefix, but Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

…lute Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

What remains is a bunch of anonymous untraceable /tmp/tmp.XXXXXXXXXX files and bak.root.receive.staff1.3835 from an error branch, testdir.1, testdir.3, and testroot454470 (with children) in testroot Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes openzfs#13259

Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259

This thoroughly destroys logapi and races to the log files horribly Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: John Kennedy <[email protected]> Reviewed-by: Ryan Moeller <[email protected]> Signed-off-by: Ahelenia Ziemiańska <[email protected]> Closes #13259

nabijaczleweli added 30 commits March 27, 2022 17:36

tests: standardise on no-arg uname with *) case for illumos

05eae1c

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: zfs_unshare_001_pos: print which filesystem failed

938d340

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: don't always skip zfs_unshare tests on FreeBSD

945cdf2

Previously, they'd all be skipped on FreeBSD where share is showmount Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: don't use share/unshare exportfs aliases, support FreeBSD NFS

8e63af2

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

zfs: simplify usage_prop_cb values

29d947b

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

zfs, zpool: safe_malloc() duplicate argv

bd2198e

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: zfs_unshare_006: log_unsupported iff usershares are actually off

97701b9

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: zfs_share_concurrent_shares: don't use log_musts in subprocesses

59ef554

This thoroughly destroys logapi and races to the log files horribly Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: nonspecific cleanup

a85030c

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: don't fail if no fio or python3.sysctl

10ba242

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: mixed_create_failure: undefined log_err -> log_fail

9077b63

Ref: openzfs#13215 Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: zfs_create_nomount: undefined local -> typeset

8e95600

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: snapshot_00[15]_pos: use cksum instead of sum -r

2ce5d74

This only worked by accident on FreeBSD, where sum doesn't take flags POSIX guarantees us cksum (Ethernet CRC) ‒ use that instead Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: replace sum(1) with cksum(1)

bdb0ca6

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: redacted_send: explicitly assume instant /dev on non-Linux

fa07bcb

The users spew udevadm ENOENTs on FreeBSD Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: vdev_zaps_007: log_must with > must eval

278f1da

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: don't >-redirect without eval

c052939

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: get rid of which

a2e4662

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: vdev_zaps_007: actually test the new pool

7ec4171

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: vdev_zaps: cleanup library

78b543f

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: README: note that -d *must* be 777 for the deleg tests

e85a827

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: README: note non-default mountd requirement

d8374eb

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: zfs_rollback_commit: talkative failures

44ba8ed

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: review every awk(1) invocation

43cb8be

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: nawk -> awk

7c6730e

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

egrep -> grep -E

0c205ec

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

fgrep -> grep -F

7f45f8d

Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: {read,write}_dos_attributes: despaghettify

09f1df1

Also: actually accept all the flags in write_d_a Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: cmd: don't recurse

6eb2c40

This confers an >10x speedup on t/z-t/cmd builds (12s -> 1.1s), gets rid of 23 redundant identical automake specs and gitignores, and groups the binaries with their common headers Signed-off-by: Ahelenia Ziemiańska <[email protected]>

tests: logapi: don't cat excessively

1873fcd

This also fixes line welding in test error output Signed-off-by: Ahelenia Ziemiańska <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test clean-out from #13165 #13259

Test clean-out from #13165 #13259

nabijaczleweli commented Mar 27, 2022

Test clean-out from #13165 #13259

Test clean-out from #13165 #13259

Conversation

nabijaczleweli commented Mar 27, 2022

Motivation and Context

Description

How Has This Been Tested?

Types of changes

Checklist: