Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Naming of fat binaries for M1 (Apple silicon) #52

Closed
vweevers opened this issue Jun 12, 2021 · 9 comments
Closed

Naming of fat binaries for M1 (Apple silicon) #52

vweevers opened this issue Jun 12, 2021 · 9 comments
Labels
discussion Discussion

Comments

@vweevers
Copy link
Member

In order to support M1 (Apple silicon), addon authors can build so-called fat binaries, that work on both x86_64 and arm64. This seems like the easiest way to add M1 support. For example fsevents/fsevents#350.

It's at odds with our current folder structure though. What would the arch be in e.g. darwin-<arch>/node.napi.node?

@vweevers
Copy link
Member Author

Options:

  1. Dual arch: darwin-x64_arm64/*.node. It'll be up to the addon author to setup their CI to produce this folder name.
  2. Some special name like darwin-universal/*.node.
  3. Same as today: darwin-x64/*.node. As a temporary solution, node-gyp-build could attempt to load this prebuild if os.arch() === 'arm64'. If it succeeds it must mean the prebuild is a fat binary.

@mafintosh
Copy link
Collaborator

@vweevers would copying it work? it's zipped anyway.

@vweevers
Copy link
Member Author

I didn't try producing a fat binary yet but I assume it will be bigger than a single-arch binary (which on rocksdb for example is already 1.81 MB gzipped). So I would prefer not duplicating it.

@lpinca
Copy link

lpinca commented Aug 21, 2021

I'm fine with option 1.

@mafintosh
Copy link
Collaborator

yea 1) or 2) is fine with me as well.

@chenrui333
Copy link

I think option 1 is better.

@vweevers
Copy link
Member Author

Let's go for option 1 then.

I wrote that "It'll be up to the addon author to setup their CI to produce this folder name" but perhaps prebuildify can do it.

Somewhat tricky because although we currently take a custom --arch or PREBUILD_ARCH which affects the folder name, we would also forward that value to node-gyp (and thus gyp and thus binding.gyp) which will probably not like the value x64_arm64.

In addition, we do that forwarding via node-gyp --target-arch:

args.push('--target_arch=' + opts.arch)

Which I think is no longer supported in node-gyp (according to docs and a quick look at code). We should be passing --arch instead.

On the other hand, if I'm right that --target-arch is ignored (since which version?) then we can avoid this problem for now. AFAICT node-gyp has no builtin support for fat binaries anyway; addon authors have to specify -arch x86_64 -arch arm64 compiler flags in their binding.gyp.

@vweevers
Copy link
Member Author

I checked, --target-arch does still work to some extent, thanks to this, but we're missing out on earlier logic that's based on --arch (for example here and here).

The node-gyp --arch option is documented since [email protected] (nodejs/node-gyp@4ee3132) which is old enough (node 8.6.0 bundles [email protected], fx).

So I propose we replace --target-arch with --arch. Using arch.split('+')[0] as the value (note that in prebuild/node-gyp-build#40 we decided to use + as arch separator instead of _) so that we only pass the first arch to node-gyp. I will send a PR.

vweevers added a commit that referenced this issue Aug 29, 2021
Technically it supports both but --arch is the documented option,
that has more logic associated with it, while --target-arch is
merely forwarded to gyp.

In addition, handle multi-arch values which on our end dictate the
output folder but are not understood by node-gyp.

Ref #52
vweevers added a commit to prebuild/node-gyp-build that referenced this issue Sep 12, 2021
vweevers added a commit that referenced this issue Sep 24, 2021
Technically it supports both but --arch is the documented option,
that has more logic associated with it, while --target-arch is
merely forwarded to gyp.

In addition, handle multi-arch values which on our end dictate the
output folder but are not understood by node-gyp.

Ref #52
@vweevers
Copy link
Member Author

Done in node-gyp-build 4.3.0 and prebuildify 4.2.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
discussion Discussion
Projects
None yet
Development

No branches or pull requests

4 participants