mount --no-canonicalize error in the dev branch

[remyd1]

Hi,

I am using the development branch version from @dschaper at GlacialMagma. Indeed, I need the --no-privilege-elevation option in syncoid. My goal here is to delegate the snapshots responsability to some advanced users.

However, I have an error, due to mount, and the --no-canonicalize option when sending the snapshot to the remote target:

/usr/local/bin/syncoid --no-privilege-elevation --compress=lzo --debug zpool3/bgui bgui@noho-128:zpool3/bgui/20180216_snapshot     

DEBUG: SSHCMD: /usr/bin/ssh    
bgui@noho-128's password: 
DEBUG: checking availability of /usr/bin/lzop on source...
DEBUG: checking availability of /usr/bin/lzop on target...
DEBUG: checking availability of /usr/bin/lzop on local machine...
DEBUG: checking availability of /usr/bin/mbuffer on source...
DEBUG: checking availability of /usr/bin/mbuffer on target...
DEBUG: checking availability of /usr/bin/pv on local machine...
DEBUG: syncing source zpool3/bgui to target zpool3/bgui/20180216_snapshot.
DEBUG: checking to see if zpool3/bgui/20180216_snapshot on /usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128 is already in zfs receive using /usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128 /bin/ps -Ao args= ...
DEBUG: checking to see if target filesystem exists using "/usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128  /usr/local/sbin/zfs get -H name zpool3/bgui/20180216_snapshot 2>&1 |"...
DEBUG: getting list of snapshots on zpool3/bgui using   /usr/local/sbin/zfs get -Hpd 1 -t snapshot guid,creation zpool3/bgui |...
DEBUG: target zpool3/bgui/20180216_snapshot does not exist.  Finding oldest available snapshot on source zpool3/bgui ...
DEBUG: getting estimated transfer size from source  using "  /usr/local/sbin/zfs send  -nP zpool3/bgui@syncoid_tycho-126_2018-02-14:16:53:40 2>&1 |"...
DEBUG: sendsize = 45680
INFO: Sending oldest full snapshot zpool3/bgui@syncoid_tycho-126_2018-02-14:16:53:40 (~ 44 KB) to new target filesystem:
DEBUG:  /usr/local/sbin/zfs send  zpool3/bgui@syncoid_tycho-126_2018-02-14:16:53:40 | /usr/bin/pv -s 45680 | /usr/bin/lzop  | /usr/bin/mbuffer  -q -s 128k -m 16M 2>/dev/null | /usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128 ' /usr/bin/mbuffer  -q -s 128k -m 16M 2>/dev/null | /usr/bin/lzop -dfc |  /usr/local/sbin/zfs receive -F zpool3/bgui/20180216_snapshot'
DEBUG: checking to see if zpool3/bgui/20180216_snapshot on /usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128 is already in zfs receive using /usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128 /bin/ps -Ao args= ...
49,5KiO 0:00:00 [3,84MiB/s] [============================================================================================================================================] 110%            
mount: only root can use "--no-canonicalize" option
cannot mount 'zpool3/bgui/20180216_snapshot': Invalid argument
CRITICAL ERROR:  /usr/local/sbin/zfs send  zpool3/bgui@syncoid_tycho-126_2018-02-14:16:53:40 | /usr/bin/pv -s 45680 | /usr/bin/lzop  | /usr/bin/mbuffer  -q -s 128k -m 16M 2>/dev/null | /usr/bin/ssh     -S /tmp/syncoid-bgui-bgui@noho-128-1518690896 bgui@noho-128 ' /usr/bin/mbuffer  -q -s 128k -m 16M 2>/dev/null | /usr/bin/lzop -dfc |  /usr/local/sbin/zfs receive -F zpool3/bgui/20180216_snapshot' failed:  at /usr/local/bin/syncoid line 251.

I looked into the syncoid source code, but I did not find any clue on this/mount.

This user has enough right on the remote dataset:

/usr/local/sbin/zfs allow zpool3/bgui                                                                                                       
---- Permissions on zpool3 -------------------------------------------
Local+Descendent permissions:
        group zfsadmins allow,create,share
        group zfsmanagers compression,create,destroy,mount,mountpoint,receive,rollback
        group Tycho compression,create,destroy,hold,mount,mountpoint,receive,rollback,send,snapshot
        group calcul_scientifique_ compression,create,destroy,hold,mount,mountpoint,receive,rollback,send,snapshot

The user is a member of the Tycho group.

[dschaper]

I haven't updated that branch in a little while, but it's some cherry picked PRs that I forked off. If I get a chance in the next few days I'll take a look and try to make sure it's up to date and has all the fixes applied.

[remyd1]

Hi @dschaper

Maybe it is not a sanoid problem, because I tried a manual send/receive with zfs and I had a similar issue:

/usr/local/sbin/zfs send zpool3/bgui/20180215_snapshot@snap2restore | ssh tycho-126 /usr/local/sbin/zfs recv -F zpool3/bgui
bgui@tycho-126's password: 
umount: only root can use "--types" option
cannot unmount 'zpool3/bgui': umount failed
exportfs: could not open /var/lib/nfs/.etab.lock for locking: errno 13 (Permission denied)
mount: only root can use "--no-canonicalize" option
cannot mount 'zpool3/bgui/restore': Invalid argument

The exportfs error could be due to a nfs share on this zpool.

Well, I think I will stay with root snapshots for the moment. This feature seems a bit too young in sanoid and even with Zfs on Linux where the posixac are well supported only from release 0.7, which means building ZoL from source (what I did, with version 0.7.5).

Kind regards,
Rémy

[remyd1]

What is weird here is: the snapshots on the target are created. So maybe these errors are not criticals.

Best regards.

[phreaker0]

@remyd1 you are hitting an ZoL issue here: openzfs/zfs#7294, on linux mounting a dataset can only be done by the root user. You did an initial sync, which in your case will do an mount of the new target dataset when finished which fails. You can do the initial replication as root and all further replications with the delegated user.

[pstch]

@phreaker0 It should actually be possible to mount datasets as unprivileged users, as long as the mount is done in a private namespace belonging to the user. It should be possible to fix the linked issue, it's not a fundamental problem, but only an unlucky situation resulting from a specific mount argument that cannot be used by unprivileged users (--no-canonicalize).

So unprivileged replications in sanoid will work at some point.

[phreaker0]

yeah I hope that they figure out a solution.