Add Rust support #477

cecton · 2020-10-01T20:13:13Z

I just started recently a new OSS project supported by Netlify: https://yewprint.rm.rs (https://github.com/cecton/yewprint) which got approved today 🎉

I'm using Rust and the build time is not short (I don't really mind personally) but I still have build time limit for the sponsored OSS account. So here I am checking if I can help getting Rust support to the Netlify image.

There was already a PR (#320) but it seems to have been closed because of lack of attention. I want to bring this back to life and add wasm-pack and wasm-bindgen support (I suppose everybody who does Rust on Netlify are here to use WASM).

Mandatory gif:

cc @jamen @JeanBarriere @ultrasaurus

Tasks:

Bring it back to life
Update
~~Add wasm-pack and wasm-bindgen~~

Closes #293
Related to #320
Blocked by #476

If `repo/target` is ignored by the repository, it's OK to cache it.

cecton · 2020-10-03T13:31:29Z

I can't even build the branch xenial locally. There is already an issue #476

\e[1mdownloading: wasmer-linux-amd64
Latest release: 1.0.0-alpha3
Downloading archive from https://github.com/wasmerio/wasmer/releases/download/1.0.0-alpha3/wasmer-linux-amd64.tar.gz
######################################################################## 100.0%
\e[1minstalling: /opt/buildhome/.wasmer
Updating bash profile /opt/buildhome/.profile
we've added the following to your /opt/buildhome/.profile
If you have a different profile please add the following:
\e[2m
# Wasmer
export WASMER_DIR="/opt/buildhome/.wasmer"
[ -s "$WASMER_DIR/wasmer.sh" ] && source "$WASMER_DIR/wasmer.sh"
/opt/buildhome/.wasmer/bin/wasmer: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.27' not found (required by /opt/buildhome/.wasmer/bin/wasmer)
\e[1merror: wasmer was installed, but doesn't seem to be working :(
The command '/bin/sh -c curl https://get.wasmer.io -sSfL | sh' returned a non-zero code: 1
[1] [10:41:40] ~/r/build-image xenial >

calavera · 2020-10-03T18:47:27Z

Hi @cecton. It might be easier for you to wait until we release #474.

We've tested it already successfully, but we're hosting a conference next week and we want to give ourselves some time to validate it widely. It should not be more than a matter of a few weeks.

cecton · 2020-10-03T18:54:33Z

Hello 👋

Yes that was my intention! When the issue will be fixed I will complete the PR with wasm-pack and test on my own repo.

Also we might want to fix the version of Rust to avoid the same issue than wasmer.

Thanks a lot for the answer

cecton · 2020-10-04T14:30:20Z

Also we might want to fix the version of Rust to avoid the same issue than wasmer.

We actually don't need to. rustup ensures rustc is compiled for glibc 2.11+ (64-bit Linux (kernel 2.6.32+, glibc 2.11+)). See https://doc.rust-lang.org/nightly/rustc/platform-support.html

So we can just install rust stable with rustup

run-build-functions.sh

mraerino · 2020-11-03T11:00:45Z

run-build.sh

@@ -39,4 +39,6 @@ echo "Executing user command: $cmd"
 eval "$cmd"
 CODE=$?

+cache_artifacts


why are you adding this?

I don't know, I'm not the author. Should I remove it?

yeah, i'd remove it

Fixed in e4e7354

Ah funny I have re-added this!

I know why the author added this. It's because when you run the test, the cache is restored (by install_dependencies in line 29) but it is not saved after the job.

cecton · 2020-11-03T16:25:01Z

run-build-functions.sh

@@ -55,11 +55,19 @@ mkdir -p $NETLIFY_CACHE_DIR/.composer
 mkdir -p $NETLIFY_CACHE_DIR/.gimme_cache/gopath
 mkdir -p $NETLIFY_CACHE_DIR/.gimme_cache/gocache
 mkdir -p $NETLIFY_CACHE_DIR/.wasmer/cache
+mkdir -p $NETLIFY_CACHE_DIR/.cargo/registry
+mkdir -p $NETLIFY_CACHE_DIR/repo/target


This should be replaced by:

mkdir -p $NETLIFY_REPO_DIR/target

i think it can be removed entirely. cargo will create target if it doesn't exist.

Fixed in 5cd8e76

run-build-functions.sh

mraerino · 2020-12-03T23:38:07Z

or you merge master in which won't require re-approval

cecton · 2020-12-04T13:34:49Z

i think this looks great!

would you want to test this on a netlify site before we merge? i can internally set this as the build image for any site you want, you'd just have to give me a site name or id

Sure! Can you do it on yewprint? https://github.com/cecton/yewprint

The build should be fixed on master now. Unfortunately, you'll have to rebase this branch to get it green @cecton

or you merge master in which won't require re-approval

I use a merge workflow. It's fixed.

✔️ CI is green! cc @calavera @mraerino

mraerino · 2020-12-04T16:20:54Z

@cecton I updated your site to run with this PR. let us know if it works for you 🙂

cecton · 2020-12-04T18:04:12Z

I tested, works like a charm @mraerino 👌

mraerino · 2020-12-04T18:07:49Z

the team that owns the build process will give this a final look and likely release it next week. in the meantime i'll leave your site configured to use it and will switch it to the regular image once we rolled this.

cecton · 2020-12-04T18:12:40Z

the team that owns the build process will give this a final look and likely release it next week. in the meantime i'll leave your site configured to use it and will switch it to the regular image once we rolled this.

Thanks! But don't worry about that because I'm stuck with manual deployment for a little longer because of some other issues. You can change it back now if that makes your life easier @mraerino

vbrown608 · 2020-12-04T22:15:10Z

Thanks for your contribution! The changes to the image and install script look good to me. Could you update the Languages section of the included software doc? https://github.com/netlify/build-image/blob/xenial/included_software.md#languages. For the version you can say Not installed by default.

(I think Languages makes more sense than Included Software, since we don't list rvm, nvm, etc. there, but open to other ideas on this.)

We're planning to automate this (#441) but in the meantime I'm going to be the bot and note the image size change, which looks reasonable:

$ docker image inspect cecton-rust:latest --format='{{.Size}}'
4427251610
$ docker image inspect netlify/build:xenial --format='{{.Size}}'
4398783842

@rstavchansky I believe we'll also need a change to https://docs.netlify.com/configure-builds/manage-dependencies/#dependency-cache, though I don't think it needs to block merging this PR and it might even be better to run it in production as an alpha for a while before adding it to the docs site. Adding the needs docs label.

This reverts commit d66e848.

cecton · 2020-12-05T14:31:52Z

Could you update the Languages section of the included software doc? https://github.com/netlify/build-image/blob/xenial/included_software.md#languages. For the version you can say Not installed by default.

Done

(I think Languages makes more sense than Included Software, since we don't list rvm, nvm, etc. there, but open to other ideas on this.)

Fair enough

vbrown608

Looks good 🦀 ! We'll include this in the next release of the build image and make a note here when it's live. Thanks again for your work on this.

vbrown608 · 2020-12-09T01:18:58Z

This is live in our Xenial build image 👍

mraerino · 2020-12-09T12:51:13Z

@cecton i reset the build image on your site. let me know if there is anything amiss.

## Motivation At Netlify we recently introduced native Rust support in the build system: netlify/build-image#477 ## Solution This PR cleans up the Netlify build config to use a more straight-forward way of building rust docs. This also introduces a workaround for netlify/build-image#505 ## Kudos We're big fans of the `tracing` crate at Netlify and using it for many new systems recently. Very happy we can give something back! Closes #1130

Jakub Turski and others added 8 commits May 29, 2019 23:48

Install stable Rust toolchain.

3525f0c

Install Rust dependencies, build optimised binaries.

73a5e62

Remove cargo update; this would make Cargo.lock useless.

09f583f

Cache Rust compiler output too.

8399525

Bring back post-execution caching.

ece7049

Make repo/target caching optional.

68c5d2b

If `repo/target` is ignored by the repository, it's OK to cache it.

Merge branch 'rust' into rust-nocache

d927ffb

Merge commit ff8f5a3 (no conflict)

8e13758

cecton mentioned this pull request Oct 4, 2020

Deploy on netlify using GH release and/or add rust to netlify's build-image yewprint/yewprint#42

Open

cecton mentioned this pull request Oct 31, 2020

Support installing tools via Homebrew (take 2) #474

Merged

mraerino reviewed Nov 3, 2020

View reviewed changes

cecton commented Nov 3, 2020

View reviewed changes

cecton added 4 commits November 5, 2020 18:18

Removed unneeded mkdir -p

5cd8e76

Removed ... something

e4e7354

Apply suggestion

db4ab67

Removed pre-compilation step

60fd309

cecton commented Nov 5, 2020

View reviewed changes

run-build-functions.sh Outdated Show resolved Hide resolved

cecton added 6 commits November 5, 2020 18:32

Fix comment and use appropriate function

b720533

Merge commit 96a3c6f (conflicts)

c56e184

Merge commit 92f69f9 (no conflict)

c0de9b0

Merge commit 19b59c0 (conflicts)

0a8215d

Merge commit 3dbb487 (no conflict)

6d1d1f6

CLEANUP

f14eeca

calavera added the type: feature code contributing to the implementation of a feature and/or user facing functionality label Nov 5, 2020

cecton added 2 commits November 28, 2020 15:11

CLEANUP

7084515

Merge commit fa6e85e (no conflict)

d39611c

cecton mentioned this pull request Dec 4, 2020

Small nits about how to get started (tracking issue) yewprint/yewprint#73

Closed

19 tasks

Merge commit 8a965d0 (no conflict)

21ae6eb

cecton mentioned this pull request Dec 4, 2020

Test Netlify with rustup yewprint/yewprint#75

Closed

vbrown608 added the needs docs label Dec 4, 2020

Update included_software.md

7f1e907

cecton dismissed stale reviews from mraerino and calavera via 7f1e907 December 5, 2020 14:25

cecton added 2 commits December 5, 2020 15:27

Added rustup to the list of tools

d66e848

Revert "Added rustup to the list of tools"

23b30a7

This reverts commit d66e848.

vbrown608 approved these changes Dec 7, 2020

View reviewed changes

vbrown608 merged commit 53b83b6 into netlify:xenial Dec 7, 2020

vbrown608 mentioned this pull request Dec 9, 2020

Document Rust support and move out of alpha #504

Open

cecton deleted the rust branch December 9, 2020 08:38

This was referenced Dec 9, 2020

chore: Debug Netlify Build tokio-rs/tracing#1130

Closed

#477 seems to have broken cargo doc-based documentation previews #505

Closed

This was referenced Dec 9, 2020

Use native Netlify rust support mraerino/tracing#1

Closed

Fix netlify build tokio-rs/tracing#1131

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Rust support #477

Add Rust support #477

cecton commented Oct 1, 2020 •

edited

Loading

cecton commented Oct 3, 2020 •

edited

Loading

calavera commented Oct 3, 2020

cecton commented Oct 3, 2020

cecton commented Oct 4, 2020

mraerino Nov 3, 2020

cecton Nov 3, 2020

mraerino Nov 3, 2020

cecton Nov 5, 2020

cecton Nov 28, 2020

cecton Nov 3, 2020

mraerino Nov 3, 2020

cecton Nov 5, 2020

mraerino commented Dec 3, 2020

cecton commented Dec 4, 2020

mraerino commented Dec 4, 2020

cecton commented Dec 4, 2020

mraerino commented Dec 4, 2020

cecton commented Dec 4, 2020

vbrown608 commented Dec 4, 2020 •

edited

Loading

cecton commented Dec 5, 2020

vbrown608 left a comment

vbrown608 commented Dec 9, 2020

mraerino commented Dec 9, 2020

Add Rust support #477

Add Rust support #477

Conversation

cecton commented Oct 1, 2020 • edited Loading

cecton commented Oct 3, 2020 • edited Loading

calavera commented Oct 3, 2020

cecton commented Oct 3, 2020

cecton commented Oct 4, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mraerino commented Dec 3, 2020

cecton commented Dec 4, 2020

mraerino commented Dec 4, 2020

cecton commented Dec 4, 2020

mraerino commented Dec 4, 2020

cecton commented Dec 4, 2020

vbrown608 commented Dec 4, 2020 • edited Loading

cecton commented Dec 5, 2020

vbrown608 left a comment

Choose a reason for hiding this comment

vbrown608 commented Dec 9, 2020

mraerino commented Dec 9, 2020

cecton commented Oct 1, 2020 •

edited

Loading

cecton commented Oct 3, 2020 •

edited

Loading

vbrown608 commented Dec 4, 2020 •

edited

Loading