Mounting NFS in RancherOS now fails after upgrade to v0.6.1

[wicadmin]

RancherOS Version: (ros os version) v0.6.1

Where are you running RancherOS? (docker-machine, AWS, GCE, baremetal, etc.) ESXi 6

This worked in v0.4.4, not now since I have just upgraded to v0.6.1 and it fails with the following error:

mount.nfs: rpc.statd is not running but is required for remote locking.
mount.nfs: Either use '-o nolock' to keep locks local, or start statd.
mount.nfs: mounting 192.168.1.50:/share failed, reason given by server: No such file or directory

In /var/lib/rancher/conf/cloud-config.d/nfs.yml, I have:

#cloud-config
rancher:
  services_include:
    ubuntu-console: true

write_files:
  # /opt/rancher/bin/start.sh is executed on start before User Docker starts
  # /etc/rc.local is also executed on start but not guaranteed to be ran before User Docker
  - path: /opt/rancher/bin/start.sh
    permissions: "0755"
    owner: root
    content: |
      #!/bin/bash
      if ! dpkg -l | grep -q nfs-common; then
        apt-get update -qq && apt-get install -y nfs-common
      fi
      mkdir -p /share
      rpcbind
      # mount -t nfs 192.168.1.50:/share /share

[joshwget]

I think this portion is the issue.

rancher:
  services_include:
    ubuntu-console: true

Alternate consoles aren't services anymore, so you're likely still in the default console. Check out the docs on alternate consoles.

[wicadmin]

Same issue, I tried without the items you listed and also changed to the following per the link:

rancher:
  console: ubuntu

[joshwget]

Have you tried running the commands manually? It would be helpful if you could find a more specific issue. Right now troubleshooting this is pretty dependent on your particular setup and cloud-config.

[wicadmin]

Yes, since they are put into the /opt/rancher/bin/start.sh, I tried to run the manually and that is where I am getting that error message. I don't seem to get an errors on boot except for the fact that the NFS are not mounted at boot time as they usually were before the upgrade. So when I run the mount commands to see what was up I get those messages.

[wicadmin]

some more info, also everytime I run the mount command, a new process is created for rpc.statd:

$ ps -ef | grep rpc.statd
statd     5728     1  0 04:38 ?        00:00:00 rpc.statd --no-notify
statd     5732     1  0 04:38 ?        00:00:00 rpc.statd --no-notify
statd     5911     1  0 04:41 ?        00:00:00 rpc.statd --no-notify
statd     5915     1  0 04:41 ?        00:00:00 rpc.statd --no-notify
statd     5927     1  0 04:45 ?        00:00:00 rpc.statd --no-notify
statd     5931     1  0 04:45 ?        00:00:00 rpc.statd --no-notify

[wicadmin]

There seems to be an issue with rpcbind. After I killed -9 it and issued a /etc/init.d/rpcbind start, then I was able to mount the NFS shares. Would be nice to find out why it wasn't started correctly before. I noticed that it was running with:

$ ps -ef | grep rpcbind
root      5719     1  0 04:38 ?        00:00:00 rpcbind

and after I killed and started it, it looks like this:

$ ps -ef | grep rpcbind
root      7338     1  0 14:45 ?        00:00:00 /sbin/rpcbind -w

[wicadmin]

Well, I just did another upgrade and tried with what I stated above and it didn't work. I probably did several things that cumulatively made it work. Need to figure out what it is exactly. Also, I am getting this message when starting rpcbind:

ln: failed to create symbolic link '/run/sendsigs.omit.d/rpcbind': No such file or directory

Any ideas?

[wicadmin]

well, here is the fix for this. I started afresh again (copying over the VM before doing the update), the only changes I made were to update 0.4.4 to 0.6.1. However, when this is completed and rebooted several times, the /etc/network/interfaces file is not there. Remember also that my previous console got destroyed in the upgrade so what I did was install the following:

apt install inetutils-ping nano

which created the /etc/network/interfaces file, but nothing much inside:

$ cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

I then added the loopback and everything works now:

$ cat /etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d

auto lo
iface lo inet loopback

[janeczku]

Closing as pre-duplicate of #1601 (comment).