proposal: cmd/compile: ability to export functions as WASM exports

[x1unix]

Background

At the moment, Go programs compiled as GOOS=js GOARCH=wasm cannot directly export functions or other symbols directly as WASM exports.

The only way to export a value is to wrap it and assign as a global JS namespace object (aka window/globalThis).

By default, each WASM module produced by Go compiler exports a few functions used internally by wasm_exec.js bridge and list of those symbols are hardcoded into a linker, like wasm_export_resume.

Current State

Current FFI allows wrapping Go functions as callbacks and assign them to JS global namespace object.
Go callback can only accept and return js.Value values.

This approach might be okay for simple tasks but might introduce some drawbacks for complex cases.

• We depend on globalThis to attach Go callbacks.
• Wrapping JS value references and calling Go's JS callback is an expensive process.
• Unable to pass low-level Go native values. Only js.Value references can be passed.

Proposal

Disclaimer: I should mention that this mechanism is not a replacement for current implementation based on syscall/js but rather an advanced option for experienced developers. Casual users might still use a classic approach. Consider it as an analogue to unsafe package but for WASM calls.

I propose to add some kind of mechanism to mark Go function to be exported in WASM module. It can be a compiler directive like //go:wasmexport or a special instruction similar to CallImport for exports.

Exported function should appear in exported WebAssembly instance symbols alongside built-in exports.

// built-in exports
wasm_export_run
wasm_export_resume
wasm_export_getsp
...

// Exported function main.Foo
main·Foo

Calling Exports

Option 1 - Manual stack preparation

This option is inspired by existing low-level import mechanism used internally by wasm_exec.js.

Values should be manually prepared and pushed into Go stack before calling an export.
Access to Go's memory allocator or arena package functions might be required for complex values like strings or slices.

Call to an exported Go function should resume WASM program execution and return back control to a caller on return.

Abstract example:

package foo

//go:wasmexport
func Bar(id int) {
   fmt.Println("Foobar:", id)
}

const callBar = (id) => {
  // Obtain exported function
  const foobarFunc = go._inst.exports["foo·Bar"];
  const { getsp, mem}  = go._inst.exports;

  // Put values on stack
  const view = new DataView(mem);
  const sp = getsp();
  view.setInt32(sp + 8, id, true);

  // Call exported Go function
  foobarFunc();
}

Option 2 - Pass args directly to a function

This option is similar to previous, but allows passing arguments to a function instead of manual stack manipulation.
Scalar types can be passed as-is but complex types should be manually allocated before passing.

It is inspired by wasm-bindgen's examples.

package main

//go:wasmexport
func hello(name string) {
   fmt.Println("Hello", name)
}

const hello = (name) => {
  // Obtain exported function
  const helloFunc = go._inst.exports["main·hello"];

  // Get program memory and memory allocator to allocate memory for a string
  const { malloc, mem }  = go._inst.exports;

  // Allocate memory for characters
  const buf = new TextEncoder('utf-8').encode(name);
  const byteArrRef = malloc(buf.length);
  const memArray = new UInt8Array(mem);
  memArray.set(buf, ptr);

  // Allocate string struct
  const stringHeaderRef = malloc(16) // sizeof(reflect.StringHeader)
  const memView = new DataView(mem);
  memView.setUint32(byteArrRef, ptr) // reflect.StringHeader.Data
  memView.setUint32(byteArrRef + 8, buf.length) // reflect.StringHeader.Len

  // Call exported Go function
  helloFunc(stringHeaderRef);
}

References

• TinyGo - Exports functions from main package by default. Don't require conversion for scalar types.
• Rust's wasm-bindgen - Functions can be exported with a special directive. Also provides access to malloc for complex values allocation.

[ianlancetaylor]

CC @golang/runtime @golang/wasm

[cherrymui]

See also #25612. Does this differ from #25612 (besides the pragma syntax)? Thanks.

[x1unix]

@cherrymui unlike mentioned issues, my proposal provides a (vague) example of desired Go function call implementation from JS/host side.

Also this issue mentions ability to access Go memory allocator on JS side for complex Go values allocation such as slices or strings.

[cherrymui]

Values should be manually prepared and pushed into Go stack before calling an export.

I'm not sure this is a good idea. This requires user code to understand the details of Go's stack-based calling convention and it is complex to write.

If we do this, I'd think the functions on the JS/Wasm side to use Wasm calling convention, so they can be called more easily on the JS/Wasm side. We can generate adaptor functions as needed. Not really sure about complex data types. Maybe wrap in a JS Value somehow.

[x1unix]

@cherrymui I was thinking about low-level foundation to start with for two reasons:

• I believe that the rest can be achieved with code generation in the same way as protoc for FlatBuffers or gRPC.
• To leave control for a user and give ability to implement more complex cases in the same manner as current WASM imports does (probably unintentionally).

As an example, I started writing a small handy library to read and write Go values from WASM memory which is used at the moment in custom WASM import functions.

I would like to avoid js.Value usage as much as possible due to produced overhead.

Also I would mention that this mechanism is not a replacement for syscall/js call but rather an advanced option for experienced developers. Casual users might still use a classic approach.

[x1unix]

I will explain my current problem to illustrate a desired behaviour.

I'm working on alternative Go playground with ability to run Go code offline right in a browser.
Internet is necessary only to download third-party dependencies.

Download dependencies are cached in IndexedDB.

Most of my logic is inside my Go program except an adapter on JS side which reads files from IndexedDB.
I was able to implement a lightweight streaming read logic using CallImport and writing Go file contents directly to a []byte slice by accessing to WebAssembly memory.

Right now there is a problem that most of browser operations are async and to pass a Go callback as a parameter, it's still necessary to wrap it using js.FuncOf and it can only accept js.Value values.

[x1unix]

@cherrymui updated proposal with a second more convenient option

[johanbrandhorst]

This also seems a dupe of #42372. I would suggest making these suggestions in that thread. I think this will have the same problems as pointed out by Richard in #42372 (comment).

[gedw99]

I see the problem

https://github.com/hack-pad/safejs Is a solution to avoid js calls.

It api direct match and so can be used easily.

There is a perf advantage with this too I found that varies be pending on the code use case..

please let me know if this helps.

[johanbrandhorst]

@x1unix Can you explain how this is different from #42372?

[x1unix]

@johanbrandhorst I gave multiple examples of possible implementation in the issue description, unlike the mentioned issue.

[johanbrandhorst]

I don't think this proposal can get to the implementation planning stage until the basic questions outlined in the other proposal about the same functionality is answered. What happens if the exported function blocks? What happens if the exported function creates a goroutine? I still think this is a duplicate of the basic functionality of #42372. I would prefer to close this issue and resume this discussion there. Once we can find answers to basic questions like this we can start talking about implementation details.

[johanbrandhorst]

Closing as duplicate of #42372

[rsc]

This proposal is a duplicate of a previously discussed proposal, as noted above,
and there is no significant new information to justify reopening the discussion.
The issue has therefore been declined as a duplicate.
— rsc for the proposal review group