Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted #6667

Open
rosseyre opened this issue Jun 14, 2022 · 21 comments
Labels

Comments

@rosseyre
Copy link

rosseyre commented Jun 14, 2022

Nix working until a couple of days ago. Can no longer enter nix-shell. Possibly caused by recent MacOS and/or XCode update..?

Tried reinstall but exits with the following error:
error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted

Any assistance would be appreciated!

System:
System Version: macOS 12.4 (21F79)
Kernel Version: Darwin 21.5.0
MacBookPro11,5

Relevant console output:

---- Preparing a Nix volume ----------------------------------------------------
    Nix traditionally stores its data in the root directory /nix, but
    macOS now (starting in 10.15 Catalina) has a read-only root directory.
    To support Nix, I will create a volume and configure macOS to mount it
    at /nix.

~~> Configuring /etc/synthetic.conf to make a mount-point at /nix

~~> Creating a Nix volume
Volume Nix Store on disk1s7 force-unmounted

~~> Configuring /etc/fstab to specify volume mount options

~~> Configuring LaunchDaemon to mount 'Nix Store'
/Users/rosseyre/.nix-profile/bin/nix-env
error: could not set permissions on '/nix/var/nix/profiles/per-user' to 755: Operation not permitted

---- oh no! --------------------------------------------------------------------
Jeeze, something went wrong. If you can take all the output and open
an issue, we'd love to fix the problem so nobody else has this issue.

:(
@rosseyre rosseyre added the bug label Jun 14, 2022
@abathur
Copy link
Member

abathur commented Jun 14, 2022

What does dscl . -read /Groups/nixbld report?

@rosseyre
Copy link
Author

rosseyre commented Jun 14, 2022

What does dscl . -read /Groups/nixbld report?

dsAttrTypeNative:record_daemon_version: 8600000
AppleMetaNodeLocation: /Local/Default
GeneratedUID: D2180FA2-03EB-4E59-82FF-630076F97807
GroupMembers: 40960014-0024-48D3-BBF1-10EFD76A9DFA 10581E7B-F30D-4DA5-A228-1148759EEEE2 183BEE55-A2C5-4EDF-8A4E-FD9CA3E6779E 3A638C72-65B6-4B4A-92E8-BC600F5C4BF4 78F5FD30-F55E-4EF6-9C99-43A930E37234 ADC51784-72EA-4AAA-A387-2F751EF2B0B6 94C0B5AA-BDA4-46D8-8CAA-B9B58665BEE7 905DE5F3-475A-45A4-AFA0-255ED5B052CF 22F92453-3B8A-4822-A4C3-EB0A8D993BED D04E27F4-A237-40AE-B1ED-5B1FB5A4519E 4B924C63-AA3A-48B4-B853-989F6887362B 2581522C-E8B3-43C6-B2A4-802B98E8F25D 642FB221-2D40-4FA1-AB51-B92C37C9B05E 28F754B1-0F1C-45F1-B5D8-DDAA6884BDB0 EED4632B-8F9F-4679-8E81-A0420D970CB1 D03172CE-ADE0-4A51-847A-B71666C3BFCF BD69329F-3123-4F67-BE6B-76640CC50C58 DF855EF5-D0FF-4DF7-9DE9-1871C94FD37A D9712A24-DDB9-4A75-A10C-372EFFB48679 472F0D54-F3D0-4DC2-AD11-F17A3C82A8AF A89EBFBD-7C8D-4B6B-8176-1C5E8AB62C4E 8D436057-0EB5-494B-9363-E553E2008139 F9FD801A-EBEB-4A14-AAA6-156BAA11BC8B A29E8C0C-00A6-4E6E-89A4-FAA16808BA96 52E9A82E-81CC-4A33-BB54-47A869920A09 6AE01513-50BB-4F8B-B30A-E33E0141510F 1977AB42-A002-4B65-AF8F-80725596648F 4C23DDF6-B8AE-4199-B84B-B7E986DB7CCD 4536AE7C-CB74-412A-9260-C0F65D374F6F 5ECF7D65-FE8B-4483-83EB-D062A7063106 2624F42B-E157-4A4F-8FEB-8A1041334F10 24D73C4C-224E-41EE-8867-49E178740F7F
GroupMembership: _nixbld1 _nixbld2 _nixbld3 _nixbld4 _nixbld5 _nixbld6 _nixbld7 _nixbld8 _nixbld9 _nixbld10 _nixbld11 _nixbld12 _nixbld13 _nixbld14 _nixbld15 _nixbld16 _nixbld17 _nixbld18 _nixbld19 _nixbld20 _nixbld21 _nixbld22 _nixbld23 _nixbld24 _nixbld25 _nixbld26 _nixbld27 _nixbld28 _nixbld29 _nixbld30 _nixbld31 _nixbld32
PrimaryGroupID: 30000
RealName:
 Nix build group for nix-daemon
RecordName: nixbld
RecordType: dsRecTypeStandard:Groups

@abathur
Copy link
Member

abathur commented Jun 14, 2022

Hmm. At a glance that looks right. What about dscl . -read /Users/<your-username> PrimaryGroupID?

@rosseyre
Copy link
Author

Hmm. At a glance that looks right. What about dscl . -read /Users/<your-username> PrimaryGroupID?

PrimaryGroupID: 20

(thanks for helping here)

@abathur
Copy link
Member

abathur commented Jun 14, 2022

(thanks for helping here)

Hehe. Not sure we're headed anywhere fast. :)

I was asking these questions to see if your situation seemed to square with #5801, which mentioned the same error message. Your responses make me think that's not the case, though.

If you need to get it up and running, you may want to follow the full uninstall instructions before trying to reinstall it again: https://nixos.org/manual/nix/stable/installation/installing-binary.html#macos

It might be ~nice to run down whatever leftover state is causing trouble, but it could easily be a time-sink that doesn't turn up anything obvious. I wouldn't really recommend that unless you're more interested in debugging it than getting up and running.

@rosseyre
Copy link
Author

rosseyre commented Jun 14, 2022

OK. I followed the uninstall instructions and attempted a reinstall. Now receiving the following error after attempting multi-user install with:
$ sh <(curl -L https://nixos.org/nix/install) --daemon


Edit: Note r.e. uninstall procedure no.4 & 5:

    1. I was unable to locate the LABEL=Nix\040Store /nix apfs rw,nobrowse line in fstab
    1. I was unable to locate the file /etc/synthetic.conf

Output:
`~~> Creating a Nix volume

---- sudo execution ------------------------------------------------------------
I am executing:

$ sudo /usr/sbin/diskutil unmount force disk1s7

to ensure the Nix volume is not mounted

disk1s7 was already unmounted

~~> Configuring /etc/fstab to specify volume mount options

---- sudo execution ------------------------------------------------------------
I am executing:

$ sudo /usr/sbin/vifs

to add nix to fstab

vifs: editing error

---- oh no! --------------------------------------------------------------------
Jeeze, something went wrong.`

@abathur
Copy link
Member

abathur commented Jun 14, 2022

Ick. That might be an instance of the issue (hopefully) fixed in #6603 (fix not released, yet). Do you see a file at /etc/.fstab.swp?

Edit: If you do find it, I think you can remove it, follow the uninstall instructions again for good measure, and hopefully run clean.

@rosseyre
Copy link
Author

Can't seem to locate /etc/.fstab.swp

@abathur
Copy link
Member

abathur commented Jun 14, 2022

Sigh. Vim's been a pain, here.

We'll need to make it cough up the error. I don't have time to entirely break it down as I need to AFK for dinner, but basically:

  1. Using
    cat > "$SCRATCH/ex_cleanroom_wrapper" <<EOF
    #!/bin/sh
    /usr/bin/ex -u NONE -n "\$@"
    EOF
    chmod 755 "$SCRATCH/ex_cleanroom_wrapper"
    EDITOR="$SCRATCH/ex_cleanroom_wrapper" _sudo "to add nix to fstab" "$@" <<EOF
    as a reference, build a version of that cat command that'll create a file in your current directory, and then run that cat command.
  2. run the chmod command on the same file
  3. run EDITOR=<thatfile> sudo vifs
  4. After you satisfy the sudo prompt, vim will hopefully cough up an error on init that you can report back? In either case, you can q out.
  5. If it didn't show an error, run echo $? and see if it prints something other than 0.
  6. If it prints anything other than 0 but didn't state a clear error, edit <thatfile> to add the verbose -V flag to the invocation, save the file, and see if it says something sensible when you re-run step 3.

🤞

@rosseyre
Copy link
Author

  • Getting nix-fix.sh: line 1: /ex_cleanroom_wrapper: Read-only file system when running the script in no.1.
  • Getting vifs: need to run as root when running EDITOR=<thatfile> sudo vifs
  • Returning 1 with echo $?

Not sure if the above was helpful ..

@abathur
Copy link
Member

abathur commented Jun 15, 2022

  • Getting nix-fix.sh: line 1: /ex_cleanroom_wrapper: Read-only file system when running the script in no.1.

Ah, sorry--I should've come back and just written this out. I think you can run (copy/paste) this on your terminal:

cat > "ex_cleanroom_wrapper" <<-EOF 
	#!/bin/sh 
	/usr/bin/ex -u NONE -n "\$@" 
	EOF
chmod 755 "ex_cleanroom_wrapper" 
EDITOR=./ex_cleanroom_wrapper sudo vifs

@rosseyre
Copy link
Author

Thanks - that works.

No errors to report back when executing EDITOR=<thatfile> sudo vifs with absolute path.
Console reports following after q out:

chdir(/etc)
fchdir() to previous dir 

Getting vifs: editing error when using relative path (same error when attempting reinstall).

@abathur
Copy link
Member

abathur commented Jun 15, 2022

Can you repeat the vifs step, and see what running echo $? reports after you q out?

@rosseyre
Copy link
Author

Can you repeat the vifs step, and see what running echo $? reports after you q out?

0

@abathur
Copy link
Member

abathur commented Jun 15, 2022

Hmm. Not sure what to make of that. :/

To show my cards a bit, at least as far as we know, the vifs: editing error appears when the underlying EDITOR command exits with a non-zero status--and unfortunately we've had a bit of an adventure shaking out unexpected cases where ex (vim) will fail, or succeed but still silently exit with a non-zero status.

We need to shake out what it's doing, but the file we're editing (fstab) and the tool we have to use to do it (vifs) make this a little tricky to do.

I'm writing a full script that will do this more like how the installer does it--hopefully that will make it clearer what the problem is.

This script assumes that the volume (disk1s7) created in the initial installer output you provided exists; you can confirm it does by running diskutil info disk1s7.

Hopefully we'll be able to reproduce the error if you can...

  1. save this script to a file (I used ~/vifs-test.sh on my system, but you can change that...)
  2. run chmod 755 ~/vifs-test.sh to make it executable
  3. run ~/vifs-test.sh and report the output
# vifs-test.nix
NIX_ROOT=/nix
uuid="$(/System/Library/Filesystems/apfs.fs/Contents/Resources/apfs.util -k "disk1s7")"
escaped_mountpoint="${NIX_ROOT/ /'\\\'040}"

set -x
SCRATCH=$(mktemp -d "${TMPDIR:-/tmp/}tmp.XXXXXXXXXX")

cat > "$SCRATCH/ex_cleanroom_wrapper" <<EOF
#!/bin/sh
/usr/bin/ex -V -u NONE -n "\$@"
EOF

chmod 755 "$SCRATCH/ex_cleanroom_wrapper"

EDITOR="$SCRATCH/ex_cleanroom_wrapper" sudo vifs <<EOF
:a
UUID=$uuid $escaped_mountpoint apfs rw,noauto,nobrowse,suid,owners
.
:x
EOF
printf "\nvifs exited $?\n"

When I run this, the output is:

$ ~/vifs-test.sh 
+++ mktemp -d /var/folders/hg/pt5wsk1n54d1prjyg9pff5_r0000gn/T/tmp.XXXXXXXXXX
++ SCRATCH=/var/folders/hg/pt5wsk1n54d1prjyg9pff5_r0000gn/T/tmp.ECmW9BBZ7E
++ cat
++ chmod 755 /var/folders/hg/pt5wsk1n54d1prjyg9pff5_r0000gn/T/tmp.ECmW9BBZ7E/ex_cleanroom_wrapper
++ EDITOR=/var/folders/hg/pt5wsk1n54d1prjyg9pff5_r0000gn/T/tmp.ECmW9BBZ7E/ex_cleanroom_wrapper
++ sudo vifs
Password:
chdir(/etc)
fchdir() to previous dir 
"/etc/fstab" 
"/etc/fstab" 7 lines, 165 characters
Entering Ex mode.  Type "visual" to go to Normal mode.
::a
UUID=A7F5D21B-5632-447B-8EC5-D9B5911D5E77 /nix apfs rw,noauto,nobrowse,suid,owners
.
::x
"/private/etc/fstab" 
"/private/etc/fstab" 8 lines, 248 characters written++ printf '\nvifs exited 0\n'

vifs exited 0

@rosseyre
Copy link
Author

rosseyre commented Jun 15, 2022

My output looks similar:

++ mktemp -d /var/folders/6j/86wgp0897jn8wc4zz829bf880000gn/T/tmp.XXXXXXXXXX
+ SCRATCH=/var/folders/6j/86wgp0897jn8wc4zz829bf880000gn/T/tmp.HhoqVzroyJ
+ cat
+ chmod 755 /var/folders/6j/86wgp0897jn8wc4zz829bf880000gn/T/tmp.HhoqVzroyJ/ex_cleanroom_wrapper
+ EDITOR=/var/folders/6j/86wgp0897jn8wc4zz829bf880000gn/T/tmp.HhoqVzroyJ/ex_cleanroom_wrapper
+ sudo vifs
Password:
chdir(/etc)
fchdir() to previous dir 
"/etc/fstab" 
"/etc/fstab" 5 lines, 123 bytes
Entering Ex mode.  Type "visual" to go to Normal mode.
::a
UUID=47F21FA8-F0F6-49F4-BEA1-DA317730D854 /nix apfs rw,noauto,nobrowse,suid,owners
.
::x
"/private/etc/fstab" 
"/private/etc/fstab" 6 lines, 206 bytes written+ printf '\nvifs exited 0\n'

vifs exited 0

@abathur
Copy link
Member

abathur commented Jun 15, 2022

Something isn't adding up, though I'm not sure what. Just back-tracking and thinking aloud...


This was probably made moot by the first uninstall attempt, but: Going back through your first post, I think maybe pulling on the permissions/operation thread mislead us, and that this line was the thread to pull on:

/Users/rosseyre/.nix-profile/bin/nix-env

My best guess is that it was emitted by the if clause at the beginning of this section:

if type -p nix-env; then
profile_packages="$(nix-env --query --installed)"
# TODO: can probably do below faster w/ read
# intentionally unquoted string to eat whitespace in wc output
# shellcheck disable=SC2046,SC2059
if ! [ $(printf "$profile_packages" | /usr/bin/wc -l) = "0" ]; then
reminder <<EOF
Nix now supports only multi-user installs on Darwin/macOS, and your user's
Nix profile has some packages in it. These packages may obscure those in the
default profile, including the Nix this installer will add. You should
review these packages:
$profile_packages
EOF

And then I think the operation-not-permitted failure happened on the nix-env --query --installed invocation in the next line. You said you couldn't enter nix-shell and I assumed this meant nix-shell wasn't on your PATH, but perhaps you were already having a permission issue? This would explain why it found nix-env in this step, and why invoking it caused trouble.


I hate to loop, but can you:

  1. repeat the uninstall instructions again
  2. run the following commands and copy the whole shell session:
    • stat /nix /etc /etc/{fstab,synthetic.conf,.fstab.swp} ~/.nix-profile
    • if /etc/fstab was present in the previous step, the output of cat /etc/fstab
    • diskutil list
    • id
  3. Try to reinstall again

@rosseyre
Copy link
Author

Thanks for grinding away with me on this.

Pasting the console output for the uninstall procedure here as well, in case that tells us anything.

Uninstall procedure output

(base) rosseyre@Rosss-MacBook-2 / % open etc/zshrc
(base) rosseyre@Rosss-MacBook-2 / % open etc/bashrc

^ neither contained lines:

`# Nix
if [ -e '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh' ]; then
. '/nix/var/nix/profiles/default/etc/profile.d/nix-daemon.sh'
fi
End Nix

(base) rosseyre@Rosss-MacBook-2 / % sudo mv /etc/zshrc.backup-before-nix /etc/zshrc

mv: /etc/zshrc.backup-before-nix: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % sudo mv /etc/bashrc.backup-before-nix /etc/bashrc 

mv: /etc/bashrc.backup-before-nix: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % sudo launchctl unload /Library/LaunchDaemons/org.nixos.nix-daemon.plist

/Library/LaunchDaemons/org.nixos.nix-daemon.plist: No such file or directory
Unload failed: 2: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % sudo rm /Library/LaunchDaemons/org.nixos.nix-daemon.plist

rm: /Library/LaunchDaemons/org.nixos.nix-daemon.plist: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % 
(base) rosseyre@Rosss-MacBook-2 / % sudo launchctl unload /Library/LaunchDaemons/org.nixos.darwin-store.plist

/Library/LaunchDaemons/org.nixos.darwin-store.plist: No such file or directory
Unload failed: 2: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % sudo rm /Library/LaunchDaemons/org.nixos.darwin-store.plist

rm: /Library/LaunchDaemons/org.nixos.darwin-store.plist: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % sudo dscl . -delete /Groups/nixbld
for u in $(sudo dscl . -list /Users | grep _nixbld); do sudo dscl . -delete /Users/$u; done

delete: Invalid Path
<dscl_cmd> DS Error: -14009 (eDSUnknownNodeName)

"4. Edit fstab using sudo vifs to remove the line mounting the Nix Store volume on /nix, which looks like this, LABEL=Nix\040Store /nix apfs rw,nobrowse"

Found only this line in file:
UID=6AFC2DF7-46A1-4256-964A-AA62909FD918 /nix apfs rw,noauto,nobrowse,suid,owner

(base) rosseyre@Rosss-MacBook-2 / % sudo vifs
vifs: editing error
(base) rosseyre@Rosss-MacBook-2 / % sudo rm /etc/synthetic.conf
Password:
(base) rosseyre@Rosss-MacBook-2 / % sudo rm -rf /etc/nix /var/root/.nix-profile /var/root/.nix-defexpr /var/root/.nix-channels ~/.nix-profile ~/.nix-defexpr ~/.nix-channels

(base) rosseyre@Rosss-MacBook-2 / % sudo diskutil apfs deleteVolume /nix

Could not find APFS Volume /nix

Post-uninstall command console output:

(base) rosseyre@Rosss-MacBook-2 / % stat /nix /etc /etc/{fstab,synthetic.conf,.fstab.swp} ~/.nix-profile
16777220 1152921504606781440 drwxr-xr-x 2 root wheel 0 64 "Jun 13 18:51:50 2022" "Jun 13 18:51:50 2022" "Jun 13 18:51:50 2022" "Jun 13 18:51:50 2022" 4096 0 0 /nix
16777220 1152921500312779689 lrwxr-xr-x 1 root wheel 0 11 "May  9 22:30:48 2022" "May  9 22:30:48 2022" "May  9 22:30:48 2022" "May  9 22:30:48 2022" 4096 0 0x88000 /etc
16777220 70529773 -rw-r--r-- 1 root wheel 0 206 "Jun 16 11:02:05 2022" "Jun 15 19:30:43 2022" "Jun 15 19:30:43 2022" "Jun 14 21:27:56 2022" 4096 8 0 /etc/fstab
stat: /etc/synthetic.conf: stat: No such file or directory
16777220 70520285 -rw-r--r-- 1 root wheel 0 12288 "Jun 14 21:24:20 2022" "Jun 14 21:24:10 2022" "Jun 14 21:24:10 2022" "Jun 14 21:23:29 2022" 4096 24 0 /etc/.fstab.swp
stat: /Users/rosseyre/.nix-profile: stat: No such file or directory
(base) rosseyre@Rosss-MacBook-2 / % cat /etc/fstab
#
# Warning - this file should only be modified with vifs(8)
#
# Failure to do so is unsupported and may be destructive.
#
UUID=47F21FA8-F0F6-49F4-BEA1-DA317730D854 /nix apfs rw,noauto,nobrowse,suid,owners
(base) rosseyre@Rosss-MacBook-2 / % diskutil list
/dev/disk0 (internal, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      GUID_partition_scheme                        *1.0 TB     disk0
   1:                        EFI ⁨EFI⁩                     209.7 MB   disk0s1
   2:                 Apple_APFS ⁨Container disk1⁩         849.0 GB   disk0s2
                    (free space)                         151.3 GB   -

/dev/disk1 (synthesized):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:      APFS Container Scheme -                      +849.0 GB   disk1
                                 Physical Store disk0s2
   1:                APFS Volume ⁨SSD (1TB) - Data⁩        461.0 GB   disk1s1
   2:                APFS Volume ⁨Preboot⁩                 532.0 MB   disk1s2
   3:                APFS Volume ⁨Recovery⁩                1.1 GB     disk1s3
   4:                APFS Volume ⁨VM⁩                      2.1 GB     disk1s4
   5:                APFS Volume ⁨SSD (1TB)⁩               15.2 GB    disk1s5
   6:              APFS Snapshot ⁨com.apple.os.update-...⁩ 15.2 GB    disk1s5s1
   7:                APFS Volume ⁨Nix⁩                     32.8 KB    disk1s8
   8:                APFS Volume ⁨Nix Store⁩               20.5 KB    disk1s7

(base) rosseyre@Rosss-MacBook-2 / % id
uid=501(rosseyre) gid=20(staff) groups=20(staff),12(everyone),61(localaccounts),79(_appserverusr),80(admin),81(_appserveradm),98(_lpadmin),701(com.apple.sharepoint.group.1),33(_appstore),100(_lpoperator),204(_developer),250(_analyticsusers),395(com.apple.access_ftp),398(com.apple.access_screensharing),399(com.apple.access_ssh),400(com.apple.access_remote_ae)
(base) rosseyre@Rosss-MacBook-2 / % 

@rosseyre
Copy link
Author

Reinstall giving:

---- sudo execution ------------------------------------------------------------
I am executing:

    $ sudo /usr/sbin/vifs

to add nix to fstab

vifs: editing error

@abathur
Copy link
Member

abathur commented Jun 16, 2022

I do see a few things here. I think maybe we're falling through some logic holes in the uninstall directions:

  1. The line in your fstab is what the uninstall instructions mean to remove (but they have an older form of the line).
  2. Since your volume isn't mounted, the diskutil deleteVolume commands aren't working. (They should probably encourage you to make sure there isn't still a Nix Store volume left).

I'll try to remember to open a PR to update those doc issues today or maybe this weekend.

So, if you can:

  1. follow the uninstall directions again, but:
    • also rm /etc/fstab and /etc/.fstab.swp
    • instead of the deleteVolume command there, run sudo diskutil apfs deleteVolume disk1s7 and sudo diskutil apfs deleteVolume disk1s8 (unless you intentionally also have a Nix volume for some purpose?)
  2. try to install again...

@rosseyre
Copy link
Author

Success! Thanks Travis.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants