gem download fails with EPERM #47

ksperling · 2017-08-02T12:49:14Z

karsten@qualifier:~$ brew gem install papertrail
==> Using the sandbox
==> Fetching papertrail from gem source
ERROR:  While executing gem ... (Errno::EPERM)
    Operation not permitted - /Users/karsten/.gem/specs/api.rubygems.org%443/specs.4.8
==> /usr/bin/gem install /Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem --no-ri --no-rdoc --no-wrapper --no-user-install --install-dir /usr/local/
Last 15 lines from /Users/karsten/Library/Logs/Homebrew/gem-papertrail/01.gem:
2017-08-03 00:07:18 +1200

/usr/bin/gem
install
/Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem
--no-ri
--no-rdoc
--no-wrapper
--no-user-install
--install-dir
/usr/local/Cellar/gem-papertrail/0.10.3
--bindir
/usr/local/Cellar/gem-papertrail/0.10.3/bin

ERROR:  Could not find a valid gem '/Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem' (>= 0) in any repository

Looking at the source, it seems brew-gem just runs gem fetch ... from the homebrew cache directory (however it should check the exit code and not continue if the download failed)

I haven't been able to figure out why gem fetch even fails in the first place though. The permissions of .gem/specs/api.rubygems.org%443 look fine. When I cd into the homebrew cache and run fetch manually it works:

karsten@qualifier:~/Library/Caches/Homebrew$ gem fetch papertrail --version 0.10.3
Fetching: papertrail-0.10.3.gem (100%)
Downloaded papertrail-0.10.3

After this brew gem install succeeds -- it attempts the download again (which fails again), but then proceeds to install the gem:

karsten@qualifier:~/Library/Caches/Homebrew$ brew gem install papertrail
==> Using the sandbox
==> Fetching papertrail from gem source
ERROR:  While executing gem ... (Errno::EPERM)
    Operation not permitted - /Users/karsten/.gem/specs/api.rubygems.org%443/specs.4.8
Warning: Cannot verify integrity of papertrail-0.10.3.gem
A checksum was not provided for this resource
For your reference the SHA256 is: a29916a203a7960b25191f6b0bca671580285273a71f6de9455c08997fe9c5de
==> /usr/bin/gem install /Users/karsten/Library/Caches/Homebrew/papertrail-0.10.3.gem --no-ri --no-rdoc --no-wrapper --no-user-install --install-dir /usr/local/
🍺  /usr/local/Cellar/gem-papertrail/0.10.3: 157 files, 513.3KB, built in 4 seconds

karsten@qualifier:~$ gem --version
karsten@qualifier:~$ which gem
/usr/bin/gem
karsten@qualifier:~$ gem --version
2.6.12
karsten@qualifier:~$ which ruby
/usr/bin/ruby
karsten@qualifier:~$ ruby --version
ruby 2.0.0p648 (2015-12-16 revision 53162) [universal.x86_64-darwin15]
karsten@qualifier:~$ brew --version
Homebrew 1.3.0-15-g2ad03b8
Homebrew/homebrew-core (git revision 4b5dd; last commit 2017-08-02)
karsten@qualifier:~$ brew info brew-gem
brew-gem: stable 0.8.1, HEAD
Install rubygems as homebrew formulae
https://github.com/sportngin/brew-gem
/usr/local/Cellar/brew-gem/0.8.1 (10 files, 11.9KB) *
  Built from source on 2017-08-03 at 00:24:08
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/brew-gem.rb

The text was updated successfully, but these errors were encountered:

nicksieger · 2017-08-02T15:10:21Z

We have seen this happening too -- perhaps there are changes to or issues with rubygems.org?

ksperling · 2017-08-02T22:09:16Z

Possible, but somewhat unlikely given that the download failed when run from brew-gem both before and after the successful manual download. Is it possible it's some kind of environment issue like a different ruby or gem binary being run via brew-gem?

nayzak · 2017-08-03T02:57:21Z

Both system and homebrew gems are work fine when executing fetch command manually

 ~/Library/Caches/Homebrew/ /usr/local/bin/gem fetch fastlane --version 2.50.1
Downloaded fastlane-2.50.1

 ~/Library/Caches/Homebrew/ /usr/bin/gem fetch fastlane --version 2.50.1
Downloaded fastlane-2.50.1

But it's failing when calling with brew gem upgrade.

nayzak · 2017-08-03T04:39:24Z

After 2 hours of investigation I found out that problem is in Homebrew itself. It has broken something in 1.2.6 and now brew-gem has no write permissions in some places.

cd /usr/local/Homebrew/
git checkout 1.2.5
brew gem upgrade fastlane #now works fine

nayzak · 2017-08-03T05:21:27Z

Here is it.
Homebrew/brew#2898
microsoft/homebrew-mssql-release#1
Note the line from the first post
==> Using the sandbox

nayzak · 2017-08-03T05:39:10Z

A've made workaround for personal usage.
Open /usr/local/Cellar/brew-gem/0.8.1/lib/brew/gem/cli.rb in text editor, go to line 77 and replace system "brew #{command} #{filename}" with system "brew #{command} --no-sandbox #{filename}".
I'm not a Ruby developer, so I don't know the right solution how to fix this issue to make a PR.

nicksieger · 2017-08-03T14:40:12Z

Ah, right. And gem is trying to update the specs cache in ~/.gem/specs which is outside of the homebrew cellar and cache. Looking into whether we can get gem to skip writing the specs.

Fixes #47.

vdupom · 2017-09-15T10:59:11Z

Still happening to me:

$ brew gem install mailcatcher
==> Fetching mailcatcher from gem source
ERROR:  While executing gem ... (Errno::EPERM)
    Operation not permitted - /Users/luca/.gem/specs/rubygems.org%443/specs.4.8
==> /usr/bin/gem install /Users/luca/Library/Caches/Homebrew/mailcatcher-0.6.5.gem --no-ri --no-rdoc --no-wrapper --no-user-install --install-dir /usr/local/Cel
Last 15 lines from /Users/luca/Library/Logs/Homebrew/gem-mailcatcher/01.gem:
2017-09-15 12:53:16 +0200

/usr/bin/gem
install
/Users/luca/Library/Caches/Homebrew/mailcatcher-0.6.5.gem
--no-ri
--no-rdoc
--no-wrapper
--no-user-install
--install-dir
/usr/local/Cellar/gem-mailcatcher/0.6.5
--bindir
/usr/local/Cellar/gem-mailcatcher/0.6.5/bin

ERROR:  Could not find a valid gem '/Users/luca/Library/Caches/Homebrew/mailcatcher-0.6.5.gem' (>= 0) in any repository

Do not report this issue to Homebrew/brew or Homebrew/core!

My system info:

$ which ruby
/usr/local/bin/ruby
$ ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin15]
$ which gem
/usr/local/bin/gem
$ gem -v
2.6.11
$ brew --version
Homebrew 1.3.2
Homebrew/homebrew-core (git revision c79fb3; last commit 2017-09-15)
$ brew info brew-gem
brew-gem: stable 0.8.2, HEAD
Install RubyGems as Homebrew formulae
https://github.com/sportngin/brew-gem
/usr/local/Cellar/brew-gem/0.8.2 (10 files, 12.6KB) *
  Built from source on 2017-09-15 at 12:53:03
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/brew-gem.rb

guaguanco · 2017-11-05T14:30:15Z

I confirm that the issue is still there for me as well with brew-gem 0.8.3

nicksieger · 2017-11-06T15:31:46Z

@vdupom @guaguanco check that the system rubygems version is upgraded to a 2.6.x version. /usr/local/bin/ruby is only used to install brew gems if you specify the --homebrew-ruby argument. For more information about how the install is done, type brew gem formula mailcatcher to see the full homebrew formula.

guaguanco · 2017-11-06T16:16:03Z

Hi @nicksieger I get the same problem when I use the --homebrew-ruby argument. Same errors as @vdupom with /usr/local/bin/gem instead of /usr/bin/gem in the messages.

I rather not touch the OSX (sierra in my case) version of ruby and use homebrew's.

It seems that the gem file doesn't get downloaded in the right place because I cannot find it in the homebrew cache folder. In attachment the formula for bcat: brew-gem_bcat-formula.rb.txt.

nicksieger · 2017-11-06T17:56:12Z

So, this is the piece of the formula that ensures that rubygems does not try to write outside of the homebrew sandbox/cache folder:

    HOMEBREW_CACHE.cd do
      ENV['GEM_SPEC_CACHE'] = "#{HOMEBREW_CACHE}/gem_spec_cache"
      system "gem", "fetch", "bcat", "--version", resource.version
    end

Can you check if that gem_spec_cache directory was ever created? This is why I ask if somehow an older version of rubygems is used because the GEM_SPEC_CACHE environment variable is a somewhat recent addition.

guaguanco · 2017-11-06T18:22:42Z

The gem_spec_cache is there in the right place, but it contains just api.rubygems.org%443/latest_specs.4.8

I guess that you're right about the outdated gem, in the RubyGemsDownloadStrategy class there should be something like

rubybindir = '/usr/local/bin'
gem_path = "#{rubybindir}/gem"
system gem_path, "fetch", "bcat", "--version", resource.version

as in the GemBcat class. system "gem" is probably using the OSX outdated gem

nicksieger · 2017-11-06T18:42:50Z

Ah, good catch! How does #51 look?

guaguanco · 2017-11-07T09:20:42Z

#51 looks good to me, it should fix the issue.

BTW wouldn't be better to add an explicit requirement for the gem version to brew-gem and to default to homebrew ruby if the system one is too old?

nicksieger added a commit that referenced this issue Aug 3, 2017

Fix sandbox permissions with gem fetch

7e23ab0

Fixes #47.

nicksieger mentioned this issue Aug 3, 2017

Fix sandbox permissions with gem spec cache #48

Merged

2 tasks

nicksieger closed this as completed in fcd1d21 Aug 3, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gem download fails with EPERM #47

gem download fails with EPERM #47

ksperling commented Aug 2, 2017

nicksieger commented Aug 2, 2017

ksperling commented Aug 2, 2017

nayzak commented Aug 3, 2017

nayzak commented Aug 3, 2017 •

edited

Loading

nayzak commented Aug 3, 2017

nayzak commented Aug 3, 2017

nicksieger commented Aug 3, 2017

vdupom commented Sep 15, 2017

guaguanco commented Nov 5, 2017 •

edited

Loading

nicksieger commented Nov 6, 2017

guaguanco commented Nov 6, 2017

nicksieger commented Nov 6, 2017 •

edited

Loading

guaguanco commented Nov 6, 2017

nicksieger commented Nov 6, 2017

guaguanco commented Nov 7, 2017 •

edited

Loading

gem download fails with EPERM #47

gem download fails with EPERM #47

Comments

ksperling commented Aug 2, 2017

nicksieger commented Aug 2, 2017

ksperling commented Aug 2, 2017

nayzak commented Aug 3, 2017

nayzak commented Aug 3, 2017 • edited Loading

nayzak commented Aug 3, 2017

nayzak commented Aug 3, 2017

nicksieger commented Aug 3, 2017

vdupom commented Sep 15, 2017

guaguanco commented Nov 5, 2017 • edited Loading

nicksieger commented Nov 6, 2017

guaguanco commented Nov 6, 2017

nicksieger commented Nov 6, 2017 • edited Loading

guaguanco commented Nov 6, 2017

nicksieger commented Nov 6, 2017

guaguanco commented Nov 7, 2017 • edited Loading

nayzak commented Aug 3, 2017 •

edited

Loading

guaguanco commented Nov 5, 2017 •

edited

Loading

nicksieger commented Nov 6, 2017 •

edited

Loading

guaguanco commented Nov 7, 2017 •

edited

Loading