Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use-snapshots build option for cross compile support. #1852

Merged
merged 20 commits into from
Mar 4, 2019

Conversation

afinch7
Copy link
Contributor

@afinch7 afinch7 commented Feb 27, 2019

For cross compiled builds it is currently impossible to generate compatible snapshots at compile time. The snapshots need to be generated on the platform they are intended to run on.

Currently it compiles fine, but fails generating the snapshot during runtime with the following error:

{"message":"Uncaught SyntaxError: Invalid regular expression flags","sourceLine":"/deno/target/release/gen/bundle/compiler.jsvar denoMain = (function () {","scriptResourceName":"/deno/target/release/gen/bundle/main.js","lineNumber":9468,"startPosition":415154,"endPosition":415155,"errorLevel":8,"startColumn":0,"endColumn":1,"isSharedCrossOrigin":false,"isOpaque":false,"frames":[]}

It also still needs to be designed in a way that works for all cross builds. I used annotations like this just to test things.

#[cfg(target_arch = "aarch64")]

@afinch7 afinch7 mentioned this pull request Feb 27, 2019
libdeno/api.cc Show resolved Hide resolved
src/snapshot.rs Outdated Show resolved Hide resolved
@ry
Copy link
Member

ry commented Feb 28, 2019

This is great - I'd love to land this.

Am I reading this right that deno would generate a snapshot every time it starts up? This is should be changed to just startup without snapshots. We actually need this feature for warming up V8 before snapshotting to retain optimized code - so I would be happy to do the preliminary work.

It would be cool to get this building in CI so that we can retain compatibility and potentially ship a binary.

@afinch7
Copy link
Contributor Author

afinch7 commented Feb 28, 2019

I still need to add some code to cache snapshots in the .deno directory so that after the first run we can use snapshots for each subsequent run on cross compiled builds.

@ry
Copy link
Member

ry commented Feb 28, 2019

It would need to startup an emulator, I guess. I would need to research what V8 does with mksnapshot. Presumably they have a buildbot for cross compilation.

I'm not into the idea of caching the snapshot in the deno_dir - this is inefficient and adds complexity.

I'm ok if we side step the snapshot completely for arm for now, until we can figure out how to generate the snapshot at compile time. We need this functionality in other places.

@afinch7
Copy link
Contributor Author

afinch7 commented Mar 1, 2019

That should work fine, but for some reason I the use-snapshots feature is always enabled. I commented it out for now so I could test on arm.

@afinch7 afinch7 changed the title [WIP]: Snapshot generation at runtime for cross compiled builds. [WIP]: No snapshots for cross compiled builds. Mar 1, 2019
build.rs Outdated Show resolved Hide resolved
@afinch7 afinch7 changed the title [WIP]: No snapshots for cross compiled builds. No snapshots for cross compiled builds. Mar 3, 2019
@afinch7 afinch7 changed the title No snapshots for cross compiled builds. use-snapshots build option for cross compile support. Mar 3, 2019
libdeno/api.cc Outdated Show resolved Hide resolved
@afinch7
Copy link
Contributor Author

afinch7 commented Mar 4, 2019

Made the requested changes. This should be ready to go.

build.rs Outdated Show resolved Hide resolved
src/isolate.rs Outdated Show resolved Hide resolved
src/isolate.rs Outdated Show resolved Hide resolved
Copy link
Member

@ry ry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good - I have a few nits...

@afinch7
Copy link
Contributor Author

afinch7 commented Mar 4, 2019

Anything else?

Copy link
Member

@ry ry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thank you! this is great to have.

I'm have no idea how to accomplish cross-compiling in CI, but it would be a good thing to have eventually. Otherwise I can imagine the arm build will eventually get out of alignment with the rest.

@ry ry merged commit 75fe80d into denoland:master Mar 4, 2019
@kevinkassimo
Copy link
Contributor

kevinkassimo commented Mar 5, 2019

Worth noting: this commit reduces Deno binary size, while doubling warm startup time:
screen shot 2019-03-04 at 11 15 55 pm

@ry
Copy link
Member

ry commented Mar 5, 2019

Oops. Not sure why this slow down is happening - but we must revert #1885

@afinch7
Copy link
Contributor Author

afinch7 commented Mar 5, 2019

I guess I'm going to need to do some investigation on this one.

@ry
Copy link
Member

ry commented Mar 5, 2019

It seems likely that the snapshot isn’t being used on x64... but I haven’t dug in

@afinch7
Copy link
Contributor Author

afinch7 commented Mar 5, 2019

Ok first clue here.
cargo build --release
works fine target/release/deno tests/002_hello.ts ~13ms locally
./tools/build.py -C target/release
not fine same benchmark ~24ms

@afinch7
Copy link
Contributor Author

afinch7 commented Mar 5, 2019

You are in fact correct @ry, but for some reason this only happens when using build.py. I can tell by binary size the the ones built with ./tools/build.py -C target/release are not using snapshots.

@afinch7
Copy link
Contributor Author

afinch7 commented Mar 5, 2019

@ry Found a fix going to create a pr now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants