Runtime error with a tinygo built wasi binary. #8

Gaboose · 2022-01-23T18:49:22Z

Tinygo wasm binaries are instrumented with asyncify by default. If you inspect the wasm.wasm binary from "Steps to Reproduce The Problem" below with wasmer inspect wasm.wasm, you will see the exported asyncify_... functions:

Type: wasm
Size: 43.6 KB
Imports:
  Functions:
    "wasi_snapshot_preview1"."fd_write": [I32, I32, I32, I32] -> [I32]
    "env"."main.add": [I32, I32, I32, I32] -> [I32]
    "env"."main.result": [I32, I32, I32] -> []
  Memories:
  Tables:
  Globals:
Exports:
  Functions:
    "malloc": [I32] -> [I32]
    "free": [I32] -> []
    "calloc": [I32, I32] -> [I32]
    "realloc": [I32, I32] -> [I32]
    "posix_memalign": [I32, I32, I32] -> [I32]
    "aligned_alloc": [I32, I32] -> [I32]
    "malloc_usable_size": [I32] -> [I32]
    "_start": [] -> []
    "asyncify_start_unwind": [I32] -> []
    "asyncify_stop_unwind": [] -> []
    "asyncify_start_rewind": [I32] -> []
    "asyncify_stop_rewind": [] -> []
    "asyncify_get_state": [] -> [I32]
  Memories:
    "memory": not shared (2 pages..)
  Tables:
  Globals:

When I try running it with this library though, I get what looks like a go panic: panic: runtime error: deadlocked: no event source is printed to stdout, and the interpreter throws a runtime error.

I'm not really sure if it's the fault with this library, tinygo or my attempt to put these two together. My use case is to run wasi compiled go code on the browser where fd_write and fd_read can return asynchronously. I'm trying to write a library where stdin and stdout are exposed as ReadableStreams, and it would be amazing if inside fd_read an async call like let chunk = await stdin.getReader().read() could work.

Expected Behavior

I get this if I remove the async keywords from the lines "main.add": async function(x, y) { and "main.result": async function(z) { lines, then everything works fine.

Actual Behavior

This is what I actually get using async imported functions.

Steps to Reproduce the Problem

Create main.go:

package main

func main() {
	result(add(2, 2))
}

func add(x, y int) int
func result(z int)

Create index.html:

<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8"/>
</head>
<body>
<script type="module">
import * as Asyncify from 'https://unpkg.com/asyncify-wasm?module';

async function main() {
    let { instance } = await Asyncify.instantiateStreaming(fetch("/wasm.wasm"), {
        wasi_snapshot_preview1: {
            fd_write: function(fd, iovsPtr, iovsLen, nwrittenPtr){
                let fullArr = [];
                let view = new DataView(instance.exports.memory.buffer);
                for (let i = 0; i < iovsLen; i++) {
                    let iovPtr = iovsPtr + i * 8;
                    let bufPtr = view.getUint32(iovPtr, true);
                    var bufLen = view.getUint32(iovPtr + 4, true);
                    let arr = new Uint8Array(view.buffer, bufPtr, bufLen);
                    for (let j = 0; j < arr.byteLength; j++) {
                        fullArr.push(arr[j]);
                    }
                }
                console.log("fd_write:", fd, String.fromCharCode.apply(null, fullArr));
                view.setUint32(nwrittenPtr, fullArr.length, true);
                return 0;
            },
        },
        env: {
            "main.add": async function(x, y) {
                console.log("add:", x, y);
                return x + y
            },
            "main.result": async function(z) {
                console.log("result:", z);
            }
        }
    });

    instance.exports._start();
}

main();
</script>
</body>
</html>

Compile main.go to a wasi target: tinygo build -o wasm.wasm -target wasi main.go.
Serve directory on http: python3 -m http.server.
Go to localhost:8000, open browser console.

Specifications

Version: https://unpkg.com/[email protected]/dist/asyncify.mjs?module
Platform: Google Chrome Version 97.0.4692.99 (Official Build) (64-bit)

The text was updated successfully, but these errors were encountered:

RReverser · 2022-01-24T00:45:52Z

My use case is to run wasi compiled go code on the browser where fd_write and fd_read can return asynchronously. I'm trying to write a library where stdin and stdout are exposed as ReadableStreams, and it would be amazing if inside fd_read an async call like let chunk = await stdin.getReader().read() could work.

That should definitely be doable, that's very similar to what https://github.com/GoogleChromeLabs/wasi-fs-access does (minus the ReadableStream but plus File System Access) and it's also based on this library.

That "unreachable" error usually happens when Asyncify runs out of its allocated stack space. In this library it's hardcoded to a fairly small number. There was an attempt to allow configuration of stack space in #5 but author has stopped responding at some point.

Meanwhile, however, can you try building with optimizations enabled? Those usually significantly reduce the required stack space, which, correspondingly, should also fix the problem. I'm not too familiar with TinyGo, but from the docs sounds like you should just pass -opt=2 or -opt=s for those to kick in.

RReverser · 2022-01-24T00:47:06Z

Then, of course, I'm also not sure if TinyGo uses asyncify for all imports or only for goroutines, but that's probably more in your area of expertise.

Gaboose · 2022-01-24T10:45:35Z

Thanks for the reply. I tried the -opt=2, -opt=s and the other options, there was no difference. -opt=z seems to be the default, when no opt flag is provided.

There was an attempt to allow configuration of stack space in #5 but author has stopped responding at some point.

That seems important. I think tinygo puts its asyncify stack in the same place where the C stack is (tinygo-org/tinygo#1101 (comment), tinygo-org/tinygo#1101 (comment)). But I can't manage to figure out what position that is exactly.

Then, of course, I'm also not sure if TinyGo uses asyncify for all imports or only for goroutines, but that's probably more in your area of expertise.

Actually, not a lot of expertise here too. But I'm afraid you're right. I just tried compiling

...
func pinger(id int) {
	for {
		time.Sleep(time.Second)
		println("ping", id)
	}
}

func main() {
	go pinger(1)
	io.Copy(os.Stdout, os.Stdin)
}

and running with wasmer and wasmtime. The io.Copy echos stdin to stdout well, but that seems to block the pinger from running altogether. While this

...
func main() {
	go pinger(1)
	pinger(2)
}

works as expected.

Gaboose · 2022-01-24T10:53:31Z

I'm also not sure if wasmer and wasmtime call the __asyncify_* exports at all.

RReverser · 2022-01-24T14:02:26Z

Yeah wasmer and wasmtime don't support Asyncify, you need a special runtime for it such as the one this library provides.

If TinyGo uses Asyncify but doesn't instrument those WASI imports, I'm not sure what you can do here. Either look for configuration that maybe would tell it to instrument those too, or maybe running wasm-opt --asyncify ... again on the output of TinyGo would work too?

Gaboose · 2022-02-11T18:05:42Z

I found that tinygo build command has a -scheduler=none option. Building the program in the original post of this issue with that, and then running [wasm-opt --asyncify ...](https://github.com/GoogleChromeLabs/asyncify#webassembly-side) worked. The downside is you can't run any goroutines without a scheduler.

I have no idea if it's possible to make the asyncify scheduler work with goroutines and imports/exports, but -scheduler=none + wasm-opt --asyncify ... solved my issue, so I guess I'll close this.

Sorry for the bother! And I appreciate the library!

RReverser · 2022-02-11T18:43:19Z

I have no idea if it's possible to make the asyncify scheduler work with goroutines and imports/exports

You might want to raise an issue on TinyGo for that. Since they use Asyncify for goroutines anyway, it means they configure the list of imports/exports somewhere in the compiler. It shouldn't be too hard for them to add a config for extra imports/exports that should be passed to Asyncify too.

Gaboose closed this as completed Feb 11, 2022

Pl8tinium mentioned this issue Dec 9, 2023

Import node.js's WASI imports in Asyncify #9

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Runtime error with a tinygo built wasi binary. #8

Runtime error with a tinygo built wasi binary. #8

Gaboose commented Jan 23, 2022 •

edited

Loading

RReverser commented Jan 24, 2022

RReverser commented Jan 24, 2022

Gaboose commented Jan 24, 2022

Gaboose commented Jan 24, 2022 •

edited

Loading

RReverser commented Jan 24, 2022

Gaboose commented Feb 11, 2022

RReverser commented Feb 11, 2022

Runtime error with a tinygo built wasi binary. #8

Runtime error with a tinygo built wasi binary. #8

Comments

Gaboose commented Jan 23, 2022 • edited Loading

Expected Behavior

Actual Behavior

Steps to Reproduce the Problem

Specifications

RReverser commented Jan 24, 2022

RReverser commented Jan 24, 2022

Gaboose commented Jan 24, 2022

Gaboose commented Jan 24, 2022 • edited Loading

RReverser commented Jan 24, 2022

Gaboose commented Feb 11, 2022

RReverser commented Feb 11, 2022

Gaboose commented Jan 23, 2022 •

edited

Loading

Gaboose commented Jan 24, 2022 •

edited

Loading