Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

0.5.0.pre fails to install/build on Github Actions (Ruby 3.0.1) #218

Closed
tisba opened this issue Nov 9, 2021 · 24 comments
Closed

0.5.0.pre fails to install/build on Github Actions (Ruby 3.0.1) #218

tisba opened this issue Nov 9, 2021 · 24 comments

Comments

@tisba
Copy link
Collaborator

tisba commented Nov 9, 2021

I'm trying to get our larger test suite running on Apple Silicon which uses mini_racer for more than just asset compilation. While I was giving 0.5.0.pre a spin it worked great locally on my Mac mini (M1, 2020), but the build failed on our GH Actions CI.

Env:

  • Ruby 3.0.1
  • Github Runner: 2.284.0 (Ubuntu 20.04.3 LTS)

I think the relevant error is (find more logs below):

[…]
2021-11-09T19:26:26.5042674Z g++: error:
2021-11-09T19:26:26.5043793Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/gems/libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/x86_64-linux/libv8/obj/libv8_monolith.a:
2021-11-09T19:26:26.5044719Z No such file or directory
[…]
logs
[…]
2021-11-09T19:26:20.9358126Z Installing libv8-node 16.10.0.0 (x86_64-linux-musl)
2021-11-09T19:26:23.8673930Z Fetching mini_racer 0.5.0.pre
2021-11-09T19:26:23.8949075Z Installing mini_racer 0.5.0.pre with native extensions
2021-11-09T19:26:26.4999443Z Gem::Ext::BuildError: ERROR: Failed to build gem native extension.
2021-11-09T19:26:26.5000444Z 
2021-11-09T19:26:26.5001449Z current directory:
2021-11-09T19:26:26.5003062Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/gems/mini_racer-0.5.0.pre/ext/mini_racer_extension
2021-11-09T19:26:26.5004164Z /opt/hostedtoolcache/Ruby/3.0.1/x64/bin/ruby -I
2021-11-09T19:26:26.5005050Z /opt/hostedtoolcache/Ruby/3.0.1/x64/lib/ruby/3.0.0 -r
2021-11-09T19:26:26.5006003Z ./siteconf20211109-1574-yya0rf.rb extconf.rb
2021-11-09T19:26:26.5006882Z checking for -lpthread... yes
2021-11-09T19:26:26.5007430Z creating Makefile
2021-11-09T19:26:26.5007750Z 
2021-11-09T19:26:26.5008390Z current directory:
2021-11-09T19:26:26.5009453Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/gems/mini_racer-0.5.0.pre/ext/mini_racer_extension
2021-11-09T19:26:26.5010254Z make DESTDIR\= clean
2021-11-09T19:26:26.5010567Z 
2021-11-09T19:26:26.5010983Z current directory:
2021-11-09T19:26:26.5011989Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/gems/mini_racer-0.5.0.pre/ext/mini_racer_extension
2021-11-09T19:26:26.5012792Z make DESTDIR\=
2021-11-09T19:26:26.5013330Z compiling mini_racer_extension.cc
2021-11-09T19:26:26.5014478Z mini_racer_extension.cc: In function ‘void* gvl_ruby_callback(void*)’:
2021-11-09T19:26:26.5015434Z mini_racer_extension.cc:1255:92: warning: ‘VALUE
2021-11-09T19:26:26.5016318Z ruby::backward::cxxanyargs::rb_rescue2(VALUE (*)(...), VALUE, VALUE (*)(...),
2021-11-09T19:26:26.5017426Z VALUE, ...)’ is deprecated: Use of ANYARGS in this function is deprecated
2021-11-09T19:26:26.5018415Z [-Wdeprecated-declarations]
2021-11-09T19:26:26.5019204Z 1255 |             RUBY_METHOD_FUNC(rescue_callback), callback_data_value,
2021-11-09T19:26:26.5019888Z rb_eException, (VALUE)0);
2021-11-09T19:26:26.5020402Z |                                                                         
2021-11-09T19:26:26.5020897Z ^
2021-11-09T19:26:26.5021293Z In file included from
2021-11-09T19:26:26.5022177Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby/internal/anyargs.h:83,
2021-11-09T19:26:26.5022787Z from
2021-11-09T19:26:26.5023531Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby/ruby.h:24,
2021-11-09T19:26:26.5024103Z from
2021-11-09T19:26:26.5024814Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby.h:38,
2021-11-09T19:26:26.5025493Z                  from mini_racer_extension.cc:2:
2021-11-09T19:26:26.5026480Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby/backward/cxxanyargs.hpp:282:1:
2021-11-09T19:26:26.5027221Z note: declared here
2021-11-09T19:26:26.5027752Z   282 | rb_rescue2(type *q, VALUE w, type *e, VALUE r, ...)
2021-11-09T19:26:26.5028227Z       | ^~~~~~~~~~
2021-11-09T19:26:26.5029213Z mini_racer_extension.cc:1255:92: warning: ‘VALUE
2021-11-09T19:26:26.5029951Z ruby::backward::cxxanyargs::rb_rescue2(VALUE (*)(...), VALUE, VALUE (*)(...),
2021-11-09T19:26:26.5030945Z VALUE, ...)’ is deprecated: Use of ANYARGS in this function is deprecated
2021-11-09T19:26:26.5031854Z [-Wdeprecated-declarations]
2021-11-09T19:26:26.5032580Z 1255 |             RUBY_METHOD_FUNC(rescue_callback), callback_data_value,
2021-11-09T19:26:26.5033174Z rb_eException, (VALUE)0);
2021-11-09T19:26:26.5033643Z |                                                                         
2021-11-09T19:26:26.5034015Z ^
2021-11-09T19:26:26.5034380Z In file included from
2021-11-09T19:26:26.5035262Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby/internal/anyargs.h:83,
2021-11-09T19:26:26.5035889Z from
2021-11-09T19:26:26.5036736Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby/ruby.h:24,
2021-11-09T19:26:26.5037319Z from
2021-11-09T19:26:26.5038054Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby.h:38,
2021-11-09T19:26:26.5038749Z                  from mini_racer_extension.cc:2:
2021-11-09T19:26:26.5039712Z /opt/hostedtoolcache/Ruby/3.0.1/x64/include/ruby-3.0.0/ruby/backward/cxxanyargs.hpp:282:1:
2021-11-09T19:26:26.5040433Z note: declared here
2021-11-09T19:26:26.5040978Z   282 | rb_rescue2(type *q, VALUE w, type *e, VALUE r, ...)
2021-11-09T19:26:26.5041442Z       | ^~~~~~~~~~
2021-11-09T19:26:26.5042131Z linking shared-object mini_racer_extension.so
2021-11-09T19:26:26.5042674Z g++: error:
2021-11-09T19:26:26.5043793Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/gems/libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/x86_64-linux/libv8/obj/libv8_monolith.a:
2021-11-09T19:26:26.5044719Z No such file or directory
2021-11-09T19:26:26.5045327Z make: *** [Makefile:262: mini_racer_extension.so] Error 1
2021-11-09T19:26:26.5045738Z 
2021-11-09T19:26:26.5046141Z make failed, exit code 2
2021-11-09T19:26:26.5046425Z 
2021-11-09T19:26:26.5046851Z Gem files will remain installed in
2021-11-09T19:26:26.5047751Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/gems/mini_racer-0.5.0.pre
2021-11-09T19:26:26.5048614Z for inspection.
2021-11-09T19:26:26.5049044Z Results logged to
2021-11-09T19:26:26.5050049Z /home/runner/work/foo/foo/vendor/bundle/ruby/3.0.0/extensions/x86_64-linux/3.0.0/mini_racer-0.5.0.pre/gem_make.out
2021-11-09T19:26:26.5050643Z 
2021-11-09T19:26:26.5051214Z An error occurred while installing mini_racer (0.5.0.pre), and Bundler cannot
2021-11-09T19:26:26.5051802Z continue.
2021-11-09T19:26:26.5052502Z Make sure that `gem install mini_racer -v '0.5.0.pre' --source
2021-11-09T19:26:26.5053409Z 'https://rubygems.org/'` succeeds before bundling.
2021-11-09T19:26:26.5053875Z 
2021-11-09T19:26:26.5054394Z In Gemfile:
2021-11-09T19:26:26.5054802Z   mini_racer
2021-11-09T19:26:26.5532742Z Took  16.81 seconds
2021-11-09T19:26:26.5543323Z ##[endgroup]
2021-11-09T19:26:26.5553008Z ##[error]The process '/opt/hostedtoolcache/Ruby/3.0.1/x64/bin/bundle' failed with exit code 5
2021-11-09T19:26:26.5676829Z Cleaning up orphan processes

On an unrelated: mini_racer 0.5.0.pre works otherwise fine on Docker Desktop on Apple Silicon 🎉! The forking issue still persists, but otherwise it seems to work very well.

/cc @lloeki

@lloeki
Copy link
Collaborator

lloeki commented Nov 10, 2021

libv8-node-16.10.0.0-x86_64-linux-musl/vendor/v8/x86_64-linux

Looks like a case of rubygems/rubygems#3174: Ubuntu is glibc so bundler has no business installing a x86_64-linux-musl version, but does and installs it. Then it fails at extension build time because the runtime platform lookup for the static lib at x86_64-linux doesn't exist within the x86_64-linux-musl gem.

@tisba
Copy link
Collaborator Author

tisba commented Nov 10, 2021

That almost sounds like a blocker for mini_racer as well, doesn't it? /cc @SamSaffron)

@lloeki
Copy link
Collaborator

lloeki commented Nov 10, 2021

mini_racer 0.4.0 already depends on libv8-node (15.x) which uses the same gem platform scheme, there's not much that changes here.

@tisba
Copy link
Collaborator Author

tisba commented Nov 10, 2021

mini_racer 0.4.0 already depends on libv8-node (15.x) which uses the same gem platform scheme, there's not much that changes here.

sorry, you are right of corse 🤦‍♂️

@themilkman
Copy link

I find myself with the same problem on kubuntu 21.10. Is there a way to work around it (except using mini_racer 0.4.0) ? Thanks for any advice!

@lloeki
Copy link
Collaborator

lloeki commented Nov 16, 2021

There could be a way, but it would double the libv8-node gem size: publish a x86_64-linux gem that contains both the glibc and musl libv8_monolith.a (hence the size doubling), and have either Libv8::Node::Paths#object_paths return only the one for the current platform or Libv8::Node::Location::Vendor#configure select only the relevant ones.

Alternatively I have a monkeypatch || patch here but it's for a bundler 1.17 / rubygems 3.0 situation: https://gist.github.com/lloeki/12dcf61324f64a2fa1e8a8b2109c1f00

It's untested against other versions, and although it only patches rubygems it may or may not break some other bundler version's assumptions about platform object comparisons.

As to why it doesn't break 0.4.0: it seems to be dependent on the order in which linux and linux-musl results appear in the gem version search result list.

@lloeki
Copy link
Collaborator

lloeki commented Nov 16, 2021

Also, you can try updating your rubygems+bundler, if that is possible for you.

@themilkman
Copy link

Hi @lloeki, first of all thanks for your response!

Unfortunately upgrading my env didn't help:

$ gem update --system
Updating rubygems-update
Fetching rubygems-update-3.2.31.gem
Successfully installed rubygems-update-3.2.31
Installing RubyGems 3.2.31
  Successfully built RubyGem
  Name: bundler
  Version: 2.2.31
  File: bundler-2.2.31.gem
Bundler 2.2.31 installed
RubyGems 3.2.31 installed

$ bundler --version
Bundler version 2.2.31

$ gem --version
3.2.31

Although monkeypatching could work (didn't try) I really don't want to do this on our production systems.
For now, I fear I'll stick to 0.4.0 and in case I find time and maybe another solution I'll post it here.

gbp added a commit to mysociety/alaveteli that referenced this issue Nov 17, 2021
Attempting to fix issue compile mini_racer v0.5.0 on GitHub Actions.
See rubyjs/mini_racer#218
@ovsant
Copy link

ovsant commented Nov 19, 2021

I was updating gem in RoR project and was getting same error. Removing Gemfile.lock and re-installing all gems through

bundle install

resolved this problem.
tested on: ruby 2.7.4, RoR 6.1.4.1

@themilkman
Copy link

themilkman commented Nov 19, 2021

Oookay, that worked here too, thanks @Alt3r! To be honest, I did not expect it to...

I think the most interesting part is this:

 PLATFORMS
-  ruby
+  x86_64-linux

which also made things like:

-    libv8 (8.4.255.0)
-    libv8-node (15.14.0.1)
+    libv8 (8.4.255.0-x86_64-linux)
+    libv8-node (16.10.0.0-x86_64-linux)
-    mini_racer (0.4.0)
-      libv8-node (~> 15.14.0.0)
+    mini_racer (0.5.0)
+      libv8-node (~> 16.10.0.0)
-    nokogiri (1.12.5)
-      mini_portile2 (~> 2.6.1)
+    nokogiri (1.12.5-x86_64-linux)

I asked a colleague to bundle install on his Mac book, which indeed resulted in an simply extended platform in the (unchanged/not removed) Gemfile.lock

+    libv8 (8.4.255.0-universal-darwin-20)
     libv8 (8.4.255.0-x86_64-linux)
+    libv8-node (16.10.0.0-arm64-darwin)
     libv8-node (16.10.0.0-x86_64-linux)

+    nokogiri (1.12.5-arm64-darwin)
+      racc (~> 1.4)
 PLATFORMS
+  arm64-darwin-20
   x86_64-linux

It seems that the PLATFORM ruby has had unhealthy implications. Not sure what's the source of all that, I don't think it a nice behaviour but now that I know it it works fine.
Hopefully this will help someone else. If someone has a deeper explanation I'd happy to see it.

@themilkman
Copy link

themilkman commented Nov 19, 2021

Well, things get even more confusing from a end user perspective:

I have a second project which had the same problem. Now, that I run bundle update, it simply uses the local mini_racer Using mini_racer 0.5.0 (was 0.4.0) without changing the PLATFORMS section from ruby to whatever.
I guess on another machine without having 0.5.0 already installed this will cause the same problem again. For me a real bundler pitfall...

Not really related to to this, but in context of the original posts, it seems my system would use musl Using libv8-node 16.10.0.0 (x86_64-linux-musl) (was 15.14.0.1

Update: JFTR; the above mentioned "Ruby-Platform" Gemfile.lock failed as expected on deployment and I had to remove/recreate it.

@solerman
Copy link

For platform bundler issues, see here rubygems/rubygems#4269

jaynetics added a commit to jaynetics/js_regex that referenced this issue Dec 5, 2021
@wbharding
Copy link

Github CI remains unwilling to use 0.5.0 on Ruby 2.6.6, even after deleting Gemfile.lock and updating as many gems as bundler could find. Reverting to 0.4.0 allows CI to start running and our Ubuntu 18 staging server to update gems, so it would seem there continues to be some incompatibility in Ruby 2.6+mini_racer 0.5.0 on Ubuntu-based platforms, which manifests as

compiling mini_racer_extension.cc
cc1plus: warning: command line option ‘-Wimplicit-int’ is valid for C/ObjC but
not for C++
cc1plus: warning: unrecognized command line option ‘-Wno-self-assign’
cc1plus: warning: unrecognized command line option ‘-Wno-parentheses-equality’
cc1plus: warning: unrecognized command line option
‘-Wno-constant-logical-operand’
cc1plus: warning: unrecognized command line option ‘-Wno-cast-function-type’
linking shared-object mini_racer_extension.so
g++: error:
/home/runner/work/gitclear/gitclear/vendor/bundle/ruby/2.6.0/gems/libv8-node-16.10.0.0-x86_64-linux/vendor/v8/x86_64-linux/libv8/obj/libv8_monolith.a:
No such file or directory
Makefile:260: recipe for target 'mini_racer_extension.so' failed
make: *** [mini_racer_extension.so] Error 1

@lloeki
Copy link
Collaborator

lloeki commented Dec 20, 2021

I have a second project which had the same problem. Now, that I run bundle update, it simply uses the local mini_racer Using mini_racer 0.5.0 (was 0.4.0) without changing the PLATFORMS section from ruby to whatever.
I guess on another machine without having 0.5.0 already installed this will cause the same problem again. For me a real bundler pitfall...

There's been this security-related change in recent bundler (mentioned above) where you now have to do e.g bundle platform add x86_64-linux, as it's not automatic anymore to look for additional platforms. This is manual and mandatory to preclude supply-chain attacks. It's a blocking step only when deployment mode is triggered, but I venture it might still affect non-deployment mode since the lockfile would not have information about the non-added platforms and would attempt automatic stuff that would behave differently than if the platform was filled in.

@lloeki
Copy link
Collaborator

lloeki commented Dec 20, 2021

Also, see this comment I made, which should work around the issue.

@jarthod
Copy link

jarthod commented Dec 22, 2021

Thanks, for the record I was having the same issue and decided to wait until the issue settles a bit before wasting the same time as everyone else trying the same things. Now I tried again and was able to upgrade "easily" by:

  1. Upgrading bundler from 2.1.2 to 2.3.0 (not sure if that's required in the end?). I used gem install bundler and then changed the version manually in my Gemfile.lock as didn't find any official way working for me.
  2. Adding the x86_64-linux platform to my Gemfile.lock with bundle lock --add-platform x86_64-linux (before it had only ruby and would cause the download of libv8-musl version, now it has both ruby and x86_64-linux and it downloads the proper libv8-linux)

@tisba
Copy link
Collaborator Author

tisba commented Dec 22, 2021

I think I'm going to close this issue. Maybe we could add a FAQ/remark in the readme about this problem, but adding the "proper" platforms seems to do the trick more or less for everyone.

@tisba tisba closed this as completed Dec 22, 2021
@KevinColemanInc
Copy link

This randomly fails in CI with the "proper" platforms set.

PLATFORMS
  ruby
  x86_64-linux

My dev machine is an M1 macbook.

@brilnius
Copy link

brilnius commented Dec 31, 2021

Not sure this is useful, here what worked for me on Linux:

gem uninstall libv8-node  # to get rid of musl version
bundle lock --add-platform x86_64-linux
gem update --system  # not sure if necessary but I didn't try again without it
bundle update --bundler

@jrcasso
Copy link

jrcasso commented Jan 5, 2022

This took longer than I'd like to admit; I'm no bundler expert. A combination of @jarthod and @brilnius suggestions fixed this issue for me. My company supports multiple platforms, caches gems locally in vendor/cache, and was using bundler version 1.17.3. Here's what I did:

  1. Uninstall mini_racer via bundle remove mini_racer.
  2. Uninstall libv8-node via bundle remove libv8-node if it's an explicit requirement in your Gemfile. If you have multiple platforms supported in your Gemfile.lock, remove them. There should now be no trace of mini_racer or libv8-node.
  3. Ensure all of the supported platforms are listed under PLATFORM in your Gemfile.lock, using bundle lock --add-platform <platform> to add those that may be missing.
  4. Update bundler via bundle update --bundler. Your Gemfile.lock should update BUNDLED WITH from 1.17.3 to 2.1.x.
  5. Ensure you have bundler 2.3.x (or later) installed via gem install bundler
  6. Manually change BUNDLED WITH from 2.1.x to 2.3.x or later (I used the latest available, which was 2.3.4)
  7. Install mini_racer via bundle add mini_racer. This will also install its libv8-node dependencies for your local platform.
  8. Ensure all platform gems are installed into your local cache via bundle package --all-platforms.

This was all that was required. Sanity check that you're good to go with a bundle install, which should be a noop.

dragon-dxw added a commit to UKGovernmentBEIS/beis-report-official-development-assistance that referenced this issue Jan 24, 2022
rubyjs/mini_racer#218 appears to reflect
the problem that we're having on Github where mini_racer doesn't
work past version 0.4
dragon-dxw added a commit to UKGovernmentBEIS/beis-report-official-development-assistance that referenced this issue Jan 24, 2022
rubyjs/mini_racer#218 appears to reflect
the problem that we're having on Github where mini_racer doesn't
work past version 0.4
@DannyBen
Copy link

DannyBen commented Mar 3, 2022

This comment solved it for me: #218 (comment)
(without gem update --system)

@lloeki
Copy link
Collaborator

lloeki commented Mar 7, 2022

Another workaround that may help people sometimes when you can't update rubygems/bundler, install libv8-node and mini_racer before bundle install:

# match the versions in your Gemfile.lock!
gem install libv8-node:16.10.0.0 # this should pick up the one for your platform
gem install mini_racer:0.6.2 # this will find the above as a dependency and build using it
bundle install # this may stubbornly install libv8-node for linux-musl again but since mini_racer is already there and built and bundler picks it up that libv8-node is not going to be used

Since you can't use bundle show/info as bundle install has not been run yet, you can hack around to get locked gem versions automatically:

$ cat Gemfile.lock | ruby -ne 'puts $1 if /^    libv8-node \((.*)\)/'
16.10.0.0-aarch64-linux
16.10.0.0-x86_64-darwin
16.10.0.0-x86_64-linux
$ cat Gemfile.lock | ruby -ne 'puts $1 if /^    mini_racer \((.*)\)/'
0.6.2

How to use that is left as an exercise for the reader.

@mikecmpbll
Copy link

mikecmpbll commented Apr 21, 2022

for anyone else unable to get it to work, you need to clobber any installed libv8-node files/directories, as rubygems/bundler can't remove it properly on it's own. find those files with:

find / -name '*libv8-node*' 2>/dev/null

this is the only way to get it to work if you've attempted to bundle before adding the platform, in my experience.

@Faq
Copy link

Faq commented Jul 14, 2022

rails 7
ruby 3.0.4 and 3.1.2
Was running into this when on mac (none M1) build docker-compose, resolved by:

  1. from 'Gemfile.lock' file remove 'ruby' under 'PLATFORMS' section
$ gem uninstall libv8-node
$ bundle lock --add-platform x86_64-linux
$ gem update --system
$ bundle update --bundler
$ bundle install

siegfault pushed a commit to siegfault/expiration that referenced this issue Aug 12, 2022
Initially upgrading with the hopes of fixing
lemurheavy/coveralls-public#1240.

Upgrading how we handle the versioning according to
https://andycroll.com/ruby/read-ruby-version-in-your-gemfile/.

Ran into rubyjs/therubyracer#467, so I ended
up resolving it by switching to mini_racer via
rubyjs/mini_racer#218 (comment).
siegfault pushed a commit to siegfault/expiration that referenced this issue Aug 12, 2022
Initially upgrading with the hopes of fixing
lemurheavy/coveralls-public#1240.

Upgrading how we handle the versioning according to
https://andycroll.com/ruby/read-ruby-version-in-your-gemfile/.

Ran into rubyjs/therubyracer#467, so I ended
up resolving it by switching to mini_racer via
rubyjs/mini_racer#218 (comment).
siegfault pushed a commit to siegfault/expiration that referenced this issue Aug 12, 2022
Initially upgrading with the hopes of fixing
lemurheavy/coveralls-public#1240.

Upgrading how we handle the versioning according to
https://andycroll.com/ruby/read-ruby-version-in-your-gemfile/.

Ran into rubyjs/therubyracer#467, so I ended
up resolving it by switching to mini_racer via
rubyjs/mini_racer#218 (comment).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests