Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiler: link .syso files found in Go package directories #4593

Draft
wants to merge 2 commits into
base: dev
Choose a base branch
from

Conversation

ydnar
Copy link
Contributor

@ydnar ydnar commented Nov 11, 2024

This is an experiment, driven by a need to link in custom sections into the resulting .wasm binary. Using a .syso file in a package directory is a well-known way to accomplish this with other architectures in big Go.

We have a POC that proves this works: ydnar/wasi-http-go#13

Another example: https://tip.golang.org/src/crypto/internal/boring/syso/

@dgryski co-authored this live at WasmCon.

@ydnar ydnar requested review from aykevl and dgryski November 11, 2024 19:52
@aykevl
Copy link
Member

aykevl commented Nov 12, 2024

While this is one option, I don't think we should be going for this solution. The reason is that .syso files would fundamentally be difficult to support in TinyGo, because existing .syso files will be expecting to use the Go calling convention (and not the TinyGo calling convention). So it would remain a wasm-only solution, which doesn't feel right to me (it feels like the wrong tool for the job).

Do you have any objections against a special-purpose pragma for this purpose? It would basically just add the file to the linker, nothing else.

Also, since this is going to be generated code and the uglyness is hidden, I don't see any fundamental reason why the #cgo hack couldn't be used? Certainly not pretty, no, but nobody except wasm-bindgen-go developers need to care about this. And it doesn't need any TinyGo changes as far as I can see.

Among other things, this enables WebAssembly programs to link in custom sections,
which can be stored in a relocatable wasm module with a .syso suffix in a package directory.

This will simplify the build process for WASI 0.2, removing the need to run
'wasm-tools component embed' with the original source WIT files.
ydnar added a commit to bytecodealliance/go-modules that referenced this pull request Nov 30, 2024
@ydnar
Copy link
Contributor Author

ydnar commented Dec 1, 2024

The Cgo hack can’t be used because it creates a circular dependency on package syscall.

What if I limited this PR to just wasm32?

@aykevl
Copy link
Member

aykevl commented Dec 2, 2024

The Cgo hack can’t be used because it creates a circular dependency on package syscall.

Yeah, you're right.

What if I limited this PR to just wasm32?

Limiting to wasm would indeed help against my worries. Or even only accepting a single filename (I don't know, something like wasi-component.syso) so that the intended use is very clear. It's easy to lift such restrictions in the future if needed, but adding them in the future may be difficult if people started using these .syso files for other purposes.

@ydnar
Copy link
Contributor Author

ydnar commented Dec 2, 2024

One complication: -Target=wasip2 sets GOARCH=arm GOOS=linux.

This means that foo_wasm.syso would not be found.

Do you have any plans to change the GOARCH/GOOS handling for architectures that TinyGo supports that Go doesn't?

ydnar added a commit to bytecodealliance/go-modules that referenced this pull request Dec 3, 2024
ydnar added a commit to bytecodealliance/go-modules that referenced this pull request Dec 6, 2024
This enables WIT tree-shaking by world or interface, as a precursor for Go package-level component metadata.

wit/bindgen: oops

wit/bindgen: add WIT generation for each WIT interface (Go package)

Currently disabled.

wit/bindgen: cgo + linker tricks WIP

wit: fix typo

wit/bindgen: typo

wit: type aliases force transitive dependency on the dependencies of their parent interface

wit: prepare for filtering interface contents

wit: remove ConstrainTo

internal/wasm: stub linking section

wit/bindgen: generate .wasm.syso files in each Go package

Currently stubbed out (if false). Depends on tinygo-org/tinygo#4593

wit: revise package sorting algorithm

This enables wasi:http to sort before wasi:cli.

wit/bindgen: more cleanup (path -> pkgPath)

wit/bindgen: optionally generate WIT files for each Go package

cmd/wit-bindgen-go: --generate-wit option to generate WIT files for each Go package

wit/bindgen: generate synthetic worlds in the go:bindgen package namespace
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants