Failing to install Drupal on macOS High Sierra - NFS filesystem issues

[kylebrowning]

Issue Type

Support Request

Your Environment

Vagrant 1.9.8
VirtualBox 5.1.27r117576
ansible 2.3.2.0
  config file = 
  configured module search path = Default w/o overrides
  python version = 2.7.10 (default, Jul 15 2017, 17:16:57) [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)]

Your OS

• macOS High Sierra 10.13

Full console output

https://gist.github.com/kylebrowning/1703b3b4cc0d07fdd03f92895313a13b

Summary

Getting an error when attempting to install Drupal 8. I redid all of these steps from the beginning and skipped blt setup doing it all manually and found the same issues.

Setting this up with MAMP actually works as intended though.

I cant seem to figure out if the problem is with composer or DrupalVM, but wanted to see if I could get some support!

I know you're busy @geerlingguy so whenever you get time. Happy to chat somewhere else besides the issue queue too!

[ajhoddinott]

I am having the same missing "link" plugin error with a vanilla drupalvm environment after updating to MacOS High Sierra. The environment, set up following the instructions at https://www.jeffgeerling.com/blog/2017/soup-nuts-using-drupal-vm-build-local-and-prod, has been running fine under Sierra.
I initially thought the timing (with the OS update) was coincidental since I was getting errors reported by the running copy of Drupal (about a missing "integer" plugin), which the host OS surely shouldn’t affect.

[kylebrowning]

Correct. High Sierra.

…

Sent from my iPhone

On Sep 16, 2017, at 7:26 PM, ajhoddinott ***@***.***> wrote: I am having the same missing "link" plugin error with a vanilla drupalvm environment after updating to MacOS High Sierra. The environment, set up following the instructions at https://www.jeffgeerling.com/blog/2017/soup-nuts-using-drupal-vm-build-local-and-prod, has been running fine under Sierra. I initially thought the timing (with the OS update) was coincidental since I was getting errors reported by the running copy of Drupal (about a missing "integer" plugin), which the host OS surely shouldn’t affect. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[oxyc]

Possibly related hashicorp/vagrant#8788 hashicorp/vagrant#8957.

Could you try with Drupal VM master branch instead of the latest release. We changed NFS to use TCP rather than UDP which could potentially fix the issue. This is the specific commit b0fc660

[oxyc]

To use TCP with v4.6.0 you need to modify the vagrant_synced_folders:

vagrant_synced_folders:
  - local_path: .
    destination: /var/www/drupalvm
    type: nfs
    create: true
    options_override:
      nfs_udp: false

[Yaroon]

The issue has nothing to do with Vagrant I don't think. If you leave Vagrant out of the setup and boot the guest OS in VirtualBox (with correct folder sharing + mounting the NFS folder on the guest OS) the problem remains.

@kylebrowning you're probably seeing different (amount of ) files in core/lib/Drupal/Core/ on the guest OS than you are on the host OS.

Count the amount of files: ls | wc -l
Display files that appear more than once (impossible really): ls | uniq -c | grep " 2"

[ajhoddinott]

Hi @Yaroon , I can’t answer for @kylebrowning , but I have the same files in core/lib/Drupal/Core on both the guest and host. However, there are several differences in other directories in core, including 3 duplicate files.

vagrant@slc-drupal-vm:/var/www/drupal/web/core$ find . | uniq -c | grep " 2"
      2 ./lib/Drupal/Core/Entity/ContentEntityTypeInterface.php
      2 ./lib/Drupal/Component/Transliteration/data/x01.php
      2 ./modules/views/tests/modules/views_test_config/test_views/views.view.test_argument.yml

mitsuyoshi:web ajh$ diff files.host files.vm 
1872d1871
< ./lib/Drupal/Core/Entity/EntityLastInstalledSchemaRepository.php
1938a1938
> ./lib/Drupal/Core/Entity/ContentEntityTypeInterface.php
2886d2885
< ./lib/Drupal/Component/Transliteration/data/x94.php
2988a2988
> ./lib/Drupal/Component/Transliteration/data/x01.php
3364d3363
< ./profiles/standard/config/install/field.field.user.user.user_picture.yml
9084,9089d9082
< ./modules/system/tests/modules/token_test
< ./modules/system/tests/modules/token_test/token_test.info.yml
< ./modules/system/tests/modules/token_test/token_test.routing.yml
< ./modules/system/tests/modules/token_test/src
< ./modules/system/tests/modules/token_test/src/Controller
< ./modules/system/tests/modules/token_test/src/Controller/TestController.php
11811d11803
< ./modules/views/tests/modules/views_test_config/test_views/views.view.test_argument_default_fixed.yml
11849a11842
> ./modules/views/tests/modules/views_test_config/test_views/views.view.test_argument.yml

Do you have any suggestions as to the cause, or a workaround?

I’m now using the master branch with tcp as suggested by @oxyc above, and my error message has changed: the geerlingguy.drupal task now reports

Drupal\\Core\\Config\\UnmetDependenciesException: Configuration objects 
provided by <em class=\"placeholder\">standard</em> have unmet
dependencies: <em
class=\"placeholder\">core.entity_form_display.user.user.default
(field.field.user.user.user_picture)</em> in
/var/www/drupal/web/core/lib/Drupal/Core/Config/UnmetDependenciesException.php:98

[Yaroon]

I don't have a workaround yet. We use vagrant for work but I'm currently setting up projects using macOS' Apache and PHP. Currently I don't use vagrant for my work (which is not good)

So you don't have weirdness in core/lib/Drupal/Core but you have it in other folders.

There cannot be 2 files with the same name in folders, so if uniq -c reports 2 or more for a file... that's... impossible :) Drupal will not work correctly if it finds files twice, or if it still sees config .yml files that shouldn't be there... Also you're only seeing file that are actually there. obviously ls | uniq -c can't list files that just aren't there on the guest but are on the host OS. The missing dependencies or missing link errors are because of files that the guest OS' Drupal does not see. That actually present on the host OS.

Can't look into it currently... but I will later. But really, for the past 2 months... I haven't found any workaround. NFS from a Linux (Ubunty & Debian afaik) on macOS High Sierra does not work reliably.

[ajhoddinott]

@Yaroon

NFS from a Linux (Ubunty & Debian afaik) on macOS High Sierra does not work reliably.

Is it possible the issue is with NFS from an APFS volume? I thought I might try with an external disk formatted with exfat, but there’s no point if you’ve already tried and found it doesn’t help.

[Yaroon]

@ajhoddinott Maybe that's the case... I don't know. From my experience...

This works: macOS Sierra with HFS+ and NFS shared folders to Linux guest OS.
This works: macOS High Sierra with APFS and NFS shared folder mounted over the network from macOS Sierra client.
This does not work: macOS High Sierra with APFS and NFS shared folders to Linux guest OS.

So it does work on High Sierra if mounted over the network from another Mac. (wether it's mounted over the network or not shouldn't really matter to the hosts's NFS Daemon though)

[geerlingguy]

@oxyc - I should really tag a new release, shouldn't I...

[oxyc]

@geerlingguy wont solve this issue unfortunately :/

[ajhoddinott]

It seems that
macOS High Sierra with HFS+ and NFS shared folders to Linux guest OS
works OK.

I made a 10GB / Mac OS Extended (Journaled) / No Partition / Sparse Bundle image in Disk Utility, mounted it, and successfully ran vagrant up on a fresh checkout. This seems to be a valid workaround (albeit slightly inconvenient).

[kylebrowning]

Can you elaborate on what changes need to be made to Drupal vm?

…

Sent from my iPhone

On Sep 18, 2017, at 6:30 PM, ajhoddinott ***@***.***> wrote: It seems that macOS High Sierra with HFS+ and NFS shared folders to Linux guest OS works OK. I made a 10GB / Mac OS Extended (Journaled) / No Partition / Sparse Bundle image in Disk Utility, mounted it, and successfully ran vagrant up on a fresh checkout. This seems to be a valid workaround. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[ajhoddinott]

@kylebrowning In the end, I haven’t made any changes. I’ve stopped using 'master' and gone back to the release version too.
The only difference is that I’m working on an HFS+ volume instead of APFS.
To avoid the inconvenience of an external disk, I’ve made a sparse bundle disk image, and I’m using that as my working directory.

[kylebrowning]

@ajhoddinott OMG. That works thank you!

For explicit instructions, on Mac OS High Sierra, open the app Disk Utility.

Press, File -> New Image -> Blank Image.

On this screen, place the bundle wherever you want. Make sure to use the format MAC OS Extended.

I did 1gig of space, but was not sure on the exact requirements.

Open this folder via command line /Volumes/{SPARSENAME}

then do your checkout or create project there.

While this isnt ideal, at least its unblocked me!

[rtackett]

@kylebrowning @ajhoddinott THANK YOU! Your workaround worked perfectly. I thought I was going crazy!

[navneet0693]

@geerlingguy
I encountered this problem a couple of weeks ago when I upgraded to beta version of High Sierra. I was working on vagrant based project, after the upgrade the drupal installation went nuts saying that several modules were missing.

I observed that this is due sync problem b/w mac and vagrant, several files were really missing inside vagrant. After searching for a while, i found quickest workaround to change your vagrant_synced_folder type to rsync and using vagrant-gatling-rsync

I have written more about this here: https://goo.gl/73WKrn

[Yaroon]

Sorry Kyle, I was too soon to post. It's not fixed, according to me. I've deleted my comments.

[froboy]

I tested the rsync solution above and rsync took over 10 min trying to sync my project (I didn't actually let it finish). This was... undesirable.

I was able to work around this by just changing nfs to as per the DrupalVM settings:

# A list of synced folders, with the keys 'local_path', 'destination', and
# a 'type' of [nfs|rsync|smb] (leave empty for slow native shares).

So in config.yml:

-    type: nfs
+    type: 

Then vagrant reload --provision seemed to fix it for me. Hope that helps others!

[oxyc]

I tested the rsync solution above and rsync took over 10 min trying to sync my project (I didn't actually let it finish). This was... undesirable.

I use rsync for all my projects and I don't think it's ever taken more than 30sec for a complete codebase sync to finish. Note that after a composer update task I ususally run vagrant rsync manually as rsync-auto syncs are way slower when it's a significant amount of file changes.

[gideoncresswell]

I've changed to rsync but now I'm left with a terminal window or Vagrant Manager window permanently open saying it is "watching". Is there anyway to stop this?

[oleyur]

I had same issue with NFS mounting on High Sierra. I did backup of Hight Sierra system, erased hard drive with formatting to Mac OS Extended (Journaled) file system. After that I have restored backup to it. Works fine.

[chrisroberts]

Just to leave a note about using HFS. While some people are finding success using it in conjunction with NFS, all of my testing with High Sierra was using an HFS formatted disk and I could reliable reproduce the error state. YMMV of course 😃

[jzavrl]

Can confirm that @froboy comment at #1547 (comment) worked for me. I don't even notice much of a performance drop.

[infovore]

Having encountered this problem with a colleague a couple of weeks back, and explored a variety of alternatives, it's worth flagging that not all workarounds are created equal. For instance: whilst you might not see a performance hit with the regular vagrant file sync, our project really does, and the speed of NFS helps hugely.

The other aspect of NFS that many of the alternatives don't offer - notably, rsync - is bidirectionality; yes, I'm often working on a virtual machine, but sometimes I'm creating files on the VM I would like accessible to the host (for instance, creating a database migration). All the various rsync options only sync to inside the virtual machine.

[dwhoban]

Leaving this here if for clarity on solution that worked for me:

1. Create a sparsebundle
2. Add the following to box/config.yml under vagrant_synced_folders:

    create: true
    mount_options: ['nolock', 'vers=3', 'tcp', 'fsc', 'rw', 'noatime']

Like I said this worked for me.

[xmacinfo]

Same solution worked for me too.

But I used only a sparsebundle. I did not need to change the sync configurations.

[kylebrowning]

Resolved in 10.13.2 beta 1

[geerlingguy]

@kylebrowning ooh... do tell more!

[kylebrowning]

It was basically an issue with NFS on APFS they Apple finally resolved in the beta release 10.13.2 High Sierra

[elvis2]

Just wanted to let others know, the 10.13.1 build has fixed my NFS issues with DrupalVM. I am running on Vagrant 2.0.0 and DrupalVM 4.6.0.

I see others (here and Vagrant on github) stating that 10.13.2 fixed theirs. Mine is fixed with 10.13.1, but, I also did the Xcode upgrade too, which might be why my setup is now working as expected.

[elvis2]

I retract my statement. After blowing away directories and having composer rebuild them (from the host machine), I do see a handful of files don't get moved to the guest. Moving on to 10.13.2 Beta...

[jeitnier]

10.13.2 beta did not solve my issue. Creating an HFS+ partition off the main disk and relocating all my vagrant stuff to this partition as per what @ajhoddinott had recommended is the only thing that worked for me.

[ghost]

Thank you @kylebrowning! I upgraded to High Sierra, and ran into this problem. Your temporary workaround worked perfectly.

[navneet0693]

This problem is solved in 10.13.2

[ghost]

@navneet0693 Thanks, I saw that in the comments above. I prefer the sparse bundle solution versus installing a beta release (which, according to @jeitnier didn't work for him).

[danepowell]

I'm trying to find a canonical way of determining when machines are affected by this problem, since it doesn't seem to happen in all cases.

This seems to be a pretty good test: run find . -type f | wc -l in the docroot outside of the vm, then again in the docroot inside the vm, and if the numbers differ, you're affected.

[Yaroon]

@danepowell it happens on all Macs with an SSD that have macOS High Sierra installed. It does not happen on iMacs with Fusion Drives. If your Mac is SSD-only then it will have had its main partition formatted as APFS when it got upgraded to macOS 10.13 (High Sierra)

Again, for completion, it is fixed in 10.13.2 which is currently in beta.

[danepowell]

I'm using an APFS-formatted SSD, running 10.13.1, I've never encountered this bug. Running the command I posted above reveals that NFS is mounting all files exactly as expected.

[kylebrowning]

Wish I could say the same. At any rate, its fixed in 10.13.2.

[justclint]

We're using acquia/blt and Ive tried a few fixes here but they've all had drastic speed reductions. Using the sparsebundle fix #1547 (comment) works well for our blt setup without any performance loss. Will be using this till the osx fix is out.

[skuark]

Also fixed for me after installing OSX 10.13.2 update.

[geerlingguy]

Closing this since it's definitely resolved (confirmed on my own Macs as well!).

[oxyc]

Any comments on the performance? I read some comments on the vagrant issue queue about it being significantly slower.

[kylebrowning]

Ive seen no degradation in performance, but YMMV