Skip to content
This repository has been archived by the owner on Oct 26, 2021. It is now read-only.

Add wasmldr #512

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open

Add wasmldr #512

wants to merge 11 commits into from

Conversation

wgwoods
Copy link
Contributor

@wgwoods wgwoods commented Sep 30, 2021

This adds internal/wasmldr and builds it alongside the shims. With the currently-enabled features, wasmldr ends up being ~4.6MB - better than expected, but still room for improvement.

Still TODO:

  • figure out how to embed wasmldr in the main binary
  • figure out how to actually execute it
  • figure out how to read in the .wasm workload
  • hook up stdio
  • rig up some tests using .wasm modules from internal/wasmldr/fixtures
  • modify github actions so new integration tests get run

@wgwoods
Copy link
Contributor Author

wgwoods commented Oct 1, 2021

(note that the current test failures are due to a bug in upstream Rust nightly - see rust-lang/rust#89432)

wgwoods and others added 2 commits October 4, 2021 10:21
* Make `main()` return Result so we can use `?`
* Factor shim-building code out into `fn cargo_build_bin(..)`

This shouldn't change the actual behavior of build.rs, but it will make
adding new internal/embedded binaries a little easier.

Signed-off-by: Will Woods <[email protected]>
Pulled a copy of enarx-wasmldr[^1] and tweaked its Config.toml just a
tiny bit so it builds like the shims. Also added the "wasmldr" feature
to control whether it gets built or not.

[^1]: https://github.com/enarx/enarx-wasmldr/tree/819f15e

Signed-off-by: Will Woods <[email protected]>
@wgwoods wgwoods force-pushed the wasmldr branch 3 times, most recently from 37105d3 to 331a719 Compare October 4, 2021 14:30
In parallel with the Backend trait, this commit adds a Workldr trait for
handling `wasmldr` (and any future workldr implementations.)

We've also got a naive function for picking the "best" workldr (there's
only one, so that's easy) and using that for `exec` if nothing was
passed on the CLI.

Signed-off-by: Will Woods <[email protected]>
We want stdout/stderr to default to a secure channel. /dev/null is
extremely secure, but not exactly developer-friendly. (How are you
supposed to print "Hello World!" without stdout?)

So, for now, we'll default to inheriting stdio from the calling process.
But we're also going to print a warning - with stupid unicode hotdogs -
to higlight the fact that this is developer-only behavior, and that it
should definitely be removed before release.

Signed-off-by: Will Woods <[email protected]>
This commit makes wasmldr try to read the wasm module from fd3 if no
file path was given. (Since keepldr doesn't currently pass arguments to
wasmldr, this will be the default behavior.)

This means you should be able to run a .wasm module in a keep by doing:

    cargo run -- exec 3< hello.wasm

This behavior will likely change soon, but hey.. Hello World!

Signed-off-by: Will Woods <[email protected]>
`_ENARX_EXEC_LEN = 4M` isn't big enough for wasmldr, which currently
weighs in at ~4.6MB. This value is an upper limit, not a static
allocation, so we can safely bump it to 128MB.

Signed-off-by: Will Woods <[email protected]>
Rust nightly started giving  us compilation problems, so pin to a
known-good version until it's fixed/reverted.

See rust-lang/rust#89432 for details.

Signed-off-by: Will Woods <[email protected]>
@wgwoods wgwoods force-pushed the wasmldr branch 2 times, most recently from bb26350 to c288918 Compare October 4, 2021 22:38
Move commonly-used integration_tests code (assert_eq_slices(),
run_test(), and a bunch of consts) out into a common test module so that
other integration tests can use them.

Also, split the run_test() function into keepldr_exec() and
check_output(), which makes the names a little clearer and also means I
don't have to rewrite the whole thing if I want to tweak the Command
args to handle wasmldr tests.

Signed-off-by: Will Woods <[email protected]>
Currently, when wasmldr exits due to ExportNotFound, it returns an exit
code of... 101. Why 101? Great question! I have absolutely no idea.

In order to make things a bit more sensible, I've defined different exit
codes for different errors. They roughly match FreeBSD's "preferable"
exit codes, as defined in `sysexits.h` - see [sysexits(3)] for more
info.

[sysexits(3)]: https://www.freebsd.org/cgi/man.cgi?sektion=3&query=sysexits

Signed-off-by: Will Woods <[email protected]>
@wgwoods wgwoods marked this pull request as ready for review October 4, 2021 22:50
@wgwoods
Copy link
Contributor Author

wgwoods commented Oct 4, 2021

Not sure the tests are actually getting run - might need to tweak the github workflows, since they run with "--no-default-features" and don't turn "wasmldr" back on...

@wgwoods
Copy link
Contributor Author

wgwoods commented Oct 4, 2021

Blerg, the tests inside internal/wasmldr are busted now because I moved those test sources to the toplevel tests/wasm for use with the wasm integration tests. And I don't know why build.rs is failing on the kvm tests. The sgx failures are more interesting - I think that's genuine breakage, not just build-system bugs. Getting close!

We can run wasm inside keeps now - let's prove it!

Here's what this patch does:

* Move wasm sources (*.wat) from internal/wasmldr/fixtures to tests/wasm
  * Copy .wat contents into wasmldr unittests and build inside the test
  * Remove wasmldr build.rs and update Cargo.toml
* Build tests/wasm/*.wat to OUT_DIR/bin/*.wasm in build.rs
* Add tests/wasmldr_tests.rs, which runs the .wasm test binaries

The intent is that we should be adding more wasm tests over time; this
gives us a place to put them and a harness to run them.

The tricky bit was getting Rust to pass the module to the
child process, since right now the only way we have to do that is to
pass it on FD3 - but Rust doesn't have `dup2()` and it *really* wants to
set FD_CLOEXEC on everything it opens, so we have to do some unsafe
things to actually pass FDs to a child process.

This isn't pretty, but it's a start.

Signed-off-by: Will Woods <[email protected]>
* Enable `--feature=wasmldr` when running the main test action
* Add `wasmldr` to the list of internal crates to test

Signed-off-by: Will Woods <[email protected]>
@wgwoods
Copy link
Contributor Author

wgwoods commented Oct 5, 2021

Okay! Fixed up the last couple of things and now everything passes except the SGX tests, which are failing due to something WASI-specific... which probably indicates a bug in the SGX crate... which means these integration tests are already finding new bugs for us to fix!

....hooray?

@wgwoods wgwoods mentioned this pull request Oct 5, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants