NFS volume does not work (permission error?)

[maxkratz]

First of all: Backup works fine with this "basic" configuration and a normal docker volume for datastore:

version: "3.3"
services:

  server:
    image: mbentley/timemachine:latest
    environment:                                                                                                                                             
      - TM_USERNAME=maxkratz                                                                                                                                 
      - TM_GROUPNAME=maxkratz                                                                                                                                
      - TM_UID=1000                                                                                                                                          
      - TM_GID=1000                                                                                                                                          
      - PASSWORD=123
    network_mode: host
    volumes:
      # data directories
      - tmdata-nfs:/opt

      # SMB stuff
      - ./data/run-samba:/run/samba                                                                                                                          
      - ./data/var-cache-samba:/var/cache/samba                                                                                                              
      - ./data/var-lib-samba:/var/lib/samba                                                                                                                  

volumes:
  tmdata-nfs:

My dockerhost is a virtual machine and I want to mount a nfs volume for the TimeMachine backups. Therefore, I add the following part to the docker-compose.yml:

volumes:
  tmdata-nfs:
    driver_opts:
      type: "nfs"
      o: "addr=10.0.0.69,nolock,soft,rw"
      device: ":/<my-export>"

Unfortunately, TimeMachine now refuses to start the backup and the log of the container lists this:

server_1  | 2020-11-09T18:40:16.730266943Z error in mds_init_ctx for: /opt/maxkratz
server_1  | 2020-11-09T18:40:16.730287522Z _mdssvc_open: Couldn't create policy handle for TimeMachine
server_1  | 2020-11-09T18:40:24.017684511Z fruit_pread_meta_stream: Removing [CDB9C062-55B3-5C1E-A5E0-D9C32358B133.sparsebundle:AFP_AfpInfo] after short read [0]
server_1  | 2020-11-09T18:40:24.018623319Z fruit_pwrite_meta_stream: On-demand create [CDB9C062-55B3-5C1E-A5E0-D9C32358B133.sparsebundle:AFP_AfpInfo] in write failed: No such file or directory

User maxkratz exists on both the vm and the NFS server and both accounts have id 1000 and group with id 1000.
I've also tried - SET_PERMISSIONS=true as parameter, but this did not fix it.

The container is able to create the folder on my NFS share with correct user and group:

 root@phoebe - /mnt/ironwolfraidfs/timemachine: $ ls -l
drwxr-sr-x 2 maxkratz maxkratz 4096 Nov  9 19:40 maxkratz
drwxr-xr-x 2 root     root     4096 Okt 24 07:03 timemachine

This is my NFS share configuration:

/mnt/ironwolfraidfs/timemachine		10.0.0.22(rw,async,no_root_squash,no_subtree_check)

Am I forgetting anything? I don't get it ... It would be very nice if someone could give me and idea!

[mbentley]

Sorry for the delayed reply. I don't have much prior experience trying to use NFS for the backend. Some online searches show that using soft, intr, and possibly locallocks on the client might be options that may be needed from looking at http://m0agx.eu/2016/03/23/using-os-x-time-machine-with-nfs/ where it refers to backing up directly to NFS. There are also some other client mount options listed there, not sure if any of them make a difference or not but it looks like you have the NFS server options that match their example so that's helpful in probably eliminating that.

The other possibility is that some of the fruit attributes don't work with a NFS backend but that's 100% a guess as I haven't used NFS in several years.

[maxkratz]

Hi @mbentley!

Thank you for your help. Unfortunately, I wasn't able to use a NFS backend successfully. However, I've decided to setup a timemachine service without NFS as a backend in first place and change that later on.

If somebody else is using NFS as a backend for this container, please let me know :-).

My actual docker-compose.yml looks like this:

version: "3.3"
services:
  server:
    image: mbentley/timemachine:latest
    environment:
      - SET_PERMISSIONS=true
      - TM_USERNAME=maxkratz
      - TM_GROUPNAME=maxkratz
      - TM_UID=1000
      - TM_GID=1000
      - PASSWORD=123
    network_mode: host
    volumes:
      # data directories
      - tmdata-nfs:/opt
      # SMB stuff
      - ./data/run-samba:/run/samba
      - ./data/var-cache-samba:/var/cache/samba
      - ./data/var-lib-samba:/var/lib/samba

volumes:
  tmdata-nfs:
    driver_opts:
      type: "nfs"
      # http://m0agx.eu/2016/03/23/using-os-x-time-machine-with-nfs/
      #o: "addr=10.0.0.69,nolock,soft,rw,intr,rsize=8192,wsize=8192,timeo=20,retrans=3,proto=tcp"
      #o: "addr=10.0.0.69,vers=3,rw,soft,intr,rsize=32768,wsize=32768,noatime,timeo=1200,retrans=10,proto=tcp,resvport,locallocks,noacl"
      # https://smekkley.wordpress.com/2014/01/18/time-machine-via-nfs-on-mavericks/
      o: "addr=10.0.0.69,vers=3,rw,soft,intr,rsize=32768,wsize=32768,noatime,timeo=1200,retrans=10,proto=tcp,resvport,noacl"
      device: ":/mnt/ironwolfraidfs/timemachine"

[mbentley]

Recently, I added a few options that you can tweak via #70. I am not sure if there are settings like changing the value of SMB_NFS_ACES to no; not sure if that has any impact or not with NFS. I also recently added the ability to tweak the values of vfs objects in #70; again not sure if anything helps there.

[maxkratz]

Thank you again for your help!

Unfortunately, the new version of the container image did not solve the problem for me. I've also tried SMB_NFS_ACES=NO but I do not know how to tweak values of vfs objects.

This is the error (taken from Docker-Compose logs):

server_1  | 2020-12-16T12:50:01.354902654Z error in mds_init_ctx for: /opt/maxkratz
server_1  | 2020-12-16T12:50:01.354935493Z _mdssvc_open: Couldn't create policy handle for TimeMachine
server_1  | 2020-12-16T12:50:08.734051974Z query_name_response: Multiple (2) responses received for a query on subnet 10.0.0.22 for name WORKGROUP<1d>.
server_1  | 2020-12-16T12:50:08.734108543Z This response was from IP 10.0.0.29, reporting an IP address of 10.0.0.29.
server_1  | 2020-12-16T12:50:11.399634644Z fruit_pread_meta_stream: Removing [AD1CC726-E011-57CA-98E3-D51FE032164C.sparsebundle:AFP_AfpInfo] after short read [0]
server_1  | 2020-12-16T12:50:11.401899720Z fruit_pwrite_meta_stream: On-demand create [AD1CC726-E011-57CA-98E3-D51FE032164C.sparsebundle:AFP_AfpInfo] in write failed: No such file or directory

[dansherman]

I have the same error. I did some digging and found that Finder is unable to copy files to the SMB share created by this container, but if I opened up terminal I could cp things over just fine. After some research I found that the NFS implementation shipped with Ubuntu 20.04 does not support xattr. This means that Finder and Time Machine cannot create (or copy) files over because the xattr is an integral part of the operation.

Next step is to figure out how to get that support...

[mbentley]

If that would be the case, it would be more that the given backing filesystem that you're using for the time machine share doesn't support xattr as NFS is alpine based inside of the container so that part is likely not relevant.

It might be worth testing to see if the backing filesystem you're using has xattr support like so:

cd /to/path/of/persistent/data
touch xattr-test
setfattr -n user.test -v "hello" xattr-test
getfattr xattr-test
rm xattr-test

On my system, I see:

# (cd /zfs/storage/timemachine && touch xattr-test; setfattr -n user.test -v "hello" xattr-test; getfattr xattr-test; rm xattr-test)
# file: xattr-test
user.test

[dansherman]

My setup is probably more complicated than it needs to be:

Proxmox host -> Ubuntu 20.04 VM / Docker -> Timemachine

I have a ZFS pool on the Proxmox host that is shared via NFS and mounted as a volume in the Timemachine container (that's why I was thinking the ubuntu support would be required, but I still have a lot to learn about Docker).

# Snippets from docker-compose

- type: volume
  source: vbackup
  target: /opt
  volume:
    nocopy: true

vbackup:
  driver_opts:
    type: "nfs"
    o: "addr=192.168.55.50,nolock,soft,rw"
    device: ":/Vault/backup"

I ran your handy xattr test commands on the Proxmox server in the backup directory:

# touch xattr-test; setfattr -n user.test -v "hello" xattr-test; getfattr xattr-test; rm xattr-test
# file: xattr-test
user.test

This is probably beyond the scope of your work, but I appreciate the pointers!

[mbentley]

Well, I would think that the issue might be that xattr might not be set on the NFS storage where it mounts the data on your Ubuntu 20.04 VM. From Proxmox, it looks like it supports xattr just fine but I am guessing from Ubuntu it is lacking. A quick Google search leads me to believe you can't use NFS as the mounting method due to a lack of xattr support.

[dansherman]

A quick Google search leads me to believe you can't use NFS as the mounting method due to a lack of xattr support.

Yep, sure looks like it. I abandoned the Ubuntu VM and moved to docker directly on proxmox. It's not "ideal," but it sure works well now. Thanks for the slick image, it works really well!

[mbentley]

Thanks @dansherman. I'm glad that you've found it useful!

[mbentley]

OK, I have added a test at startup that will see if we can set xattrs. If not, it will just print a warning message.

[ethul]

As a follow up question to this issue, has anyone had success in using a network mount (other than nfs) for the backing filesystem that does support xattr? I was thinking of mounting a cifs network share (which supports xattr) and using that for the backing filesystem. I would be curious if anyone has gotten this working and has an example configuration. I might be interesting in trying out this docker image if this were a possibility.