Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

reduce build size #1046

Closed
wants to merge 2 commits into from
Closed

reduce build size #1046

wants to merge 2 commits into from
+13 −0

Conversation

ahdinosaur
Copy link
Contributor

@ahdinosaur ahdinosaur commented May 20, 2019 •
edited
Loading

fixes #1045 and (theoretically) reduces build size (even more!)

  • sodium-native and (now) leveldown are the only dependencies using prebuilds
  • the only sodium-native binding we need is electron-69.node
    • where $(electron --abi) == 69
  • we also need the libsodium.so.23 shared library
    • but we must copy this into the root directory of the build (outside the compiled binary file)
  • the only leveldown binding we need is node.napi.node

@ahdinosaur ahdinosaur mentioned this pull request May 20, 2019
Merged
@ahdinosaur
Copy link
Contributor Author

also worth mentioning that i expected for us to be using leveldown prebuilds, but seems we're not:

$ find node_modules -name 'prebuilds' -exec echo {} \;
node_modules/ssb-server/node_modules/sodium-native/prebuilds
node_modules/sodium-native/prebuilds

if in the future we do (or start using another module with prebuilds), we will need to explicitly add them as we have here.

@christianbundy
Copy link
Contributor

Thanks for putting this together! I tried it and saw that dist/linux-unpacked/ contained libsodium.so.23 but I couldn't find any of the prebuilds. Do you see them in that directory?

@ahdinosaur
Copy link
Contributor Author

ahdinosaur commented May 21, 2019 •
edited
Loading

@christianbundy my understanding is the prebuilds get built into the ssb-patchwork binary. or at least that's what i gathered from not a single .js or .node file being in dist/linux-unpacked/ when i started, and the debug logs gave me a fuzzy indication of walking the node graph to put everything together. do you normally see the prebuilds in dist/linux-unpacked/?

@christianbundy
Copy link
Contributor

Yeah, here's how I've been verifying them:

$ find dist/linux-unpacked -name '*.node'
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-67.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-53.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-59.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-54.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-49.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/node-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-69.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-50.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-67.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-53.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-59.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-54.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-49.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/node-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-69.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-x64/electron-50.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-67.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-53.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-59.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-54.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-49.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/node-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-69.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-arm/electron-50.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-67.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-53.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-59.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-54.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-49.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/node-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-69.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/win32-ia32/electron-50.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-67.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-53.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-57.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-59.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-54.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-49.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-64.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/node-47.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-51.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-69.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-48.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/darwin-x64/electron-50.node

@christianbundy
Copy link
Contributor

Just to clarify: that's the result from master, when I run that command on this branch I don't have any results. Maybe it's not our fault at all? https://github.com/sodium-friends/sodium-native/issues/93#issuecomment-492966737

But somehow the only way I've seen a non-broken config is leaving in all of the bloated and irrelevant prebuild files.

@ahdinosaur
fix libsodium build
6923fd6 
fixes #1045 and (theoretically) reduces build size

- sodium-native is the only dependency using prebuilds
- the only node binding we need is `electron-69.node`
  - where `$(electron --abi) == 69`
- we also need the `libsodium.so.23` shared library
  - but we must copy this into the root directory of the build (outside the compiled binary file)
@ahdinosaur
Copy link
Contributor Author

@christianbundy do you have to do something special to get dist/linux-unpacked/resources/app.asar.unpacked/? i don't see that, i'm running npm run dist -- --dir (same as electron-builder --dist) to generate the dist/linux-unpacked.

@ahdinosaur ahdinosaur force-pushed the fix/libsodium-build branch from 4121b70 to 6923fd6 Compare May 21, 2019 06:23
@ahdinosaur ahdinosaur force-pushed the fix/libsodium-build branch from 9fb8446 to c8bd891 Compare May 21, 2019 06:35
@ahdinosaur
Copy link
Contributor Author

okay, rebased, updated this to include the new leveldown prebuild, is working again for me.

today i learned about asarUnpack: "* to get dist/linux-unpacked/resources/app.asar.unpacked/

(also using asarUnpack might be another way to get the libsodium.so working, as i see Matt did this in the late `patchwork-builder: https://github.com/ssbc/patchwork-builder/blob/master/package.json#L24)

$ find dist/linux-unpacked -name '*.node'
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/sodium-native/prebuilds/linux-x64/electron-69.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/@nornagon/cld/build/Release/cld.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/keyboard-layout/build/Release/keyboard-layout-manager.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/mouse-forward-back/build/Release/mouse-forward-back.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/@felixrieseberg/spellchecker/build/Release/spellchecker.node
dist/linux-unpacked/resources/app.asar.unpacked/node_modules/leveldown/prebuilds/linux-x64/node.napi.node

@ahdinosaur
Copy link
Contributor Author

(also using asarUnpack might be another way to get the libsodium.so working, as i see Matt did this in the late `patchwork-builder: https://github.com/ssbc/patchwork-builder/blob/master/package.json#L24)

nevermind, that doesn't work for me, i get the same error

loading sodium bindings: libsodium.so.23: cannot open shared object file: No such

@christianbundy
Copy link
Contributor

FWIW, I'm just doing npm run dist -- --linux=AppImage to generate an AppImage to test.

@ahdinosaur ahdinosaur mentioned this pull request May 22, 2019
Closed
@ahdinosaur ahdinosaur changed the title fix libsodium build reduce build size May 22, 2019
@stale
Copy link

stale bot commented Aug 20, 2019

Is this still relevant? If so, what is blocking it? Is there anything you can do to help move it forward?

@stale stale bot added the stale label Aug 20, 2019
@christianbundy
Copy link
Contributor

@ahdinosaur Should this remain open?

@stale stale bot removed the stale label Oct 14, 2019
@ahdinosaur
Copy link
Contributor Author

if anyone wants to take this over, yes, but otherwise no.

@christianbundy
Copy link
Contributor

I don't think it's something I have capacity for, I'm tired of banging my head against electron-builder. Thanks for experimenting with this as well, hopefully one of us will have time for it in the future. ❤️

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers
No reviews
Assignees
No one assigned
Labels
topic:builds type:enhancement
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

error loading sodium bindings: libsodium.so.23
2 participants
@ahdinosaur @christianbundy