Consider abstract types implemented by wasm (not the host)

[lukewagner]

If WASI wants to switch from i32 handles to abstract reference types then, with the current proposal, if we want to implement this WASI interface in pure wasm (e.g., on the Web, where WASI isn't implemented natively by the host), then the implementing wasm module has two problems:

1. it can only implement the abstract reference type with an anyref, funcref or ref (func ...)
2. the exported type is not actually abstract, and thus its values can be forged

Limitation (1) means that, if you want to implement an abstract data type in linear memory, you'll have to either stuff the i32 pointer into an anyref in JS and then call out to JS to "unbox" the pointer back into an i32, or, alternatively, smuggle the i32 into a function reference via func.bind. Neither seem ideal; it seems like what we'd want is i31ref.

Fixing limitation (2) I think entails including the newtype discussed in a previous issue.

Thoughts?

[rossberg]

I agree with what you're saying, and I'm in favour of both these mechanisms. But they both are semantically independent from type imports, so would probably warrant separate proposals.

[lukewagner]

I agree they're theoretically separable, but it still might make sense to group them to have a coherent wasm language addition that can be used to solve a whole problem. Also, the exact way that newtype and type imports/exports interact to allow data hiding seems crucial to get right, so it seems risky to leave newtype for later.

[rossberg]

But what would newtype define, if not a GC type? Wouldn't that undermine the very motivation for splitting this proposal from the GC one?

[lukewagner]

But what would newtype define, if not a GC type?

As I said above, an i31 ;)

[rossberg]

Ah, I see, you want it for that type only.

[lukewagner]

I suppose technically both i31 and any (the latter being rather useful in conjunction with JS, especially soon with weak refs).

[tlively]

If we had a newtype mechanism, it would be nice if we could just newtype i32s and other numeric types into abstract exported types rather than going through something like an i32ref. Would that be on the table?

[rossberg]

@tlively, unfortunately, I don't think that is possible. It must be interchangeable with other references like externref, e.g., to be usable as a type import. And you'll inversely want to be able to export such types (or values thereof) to JS without breaking abstraction. So some boxing to protect it is unavoidable, AFAICS.

[tlively]

It would be nice if type imports could be bound by the type of the underlying representation without having a subtype relationship with the underlying type, so the engine could know to use an i32 internally but abstraction would be preserved by validation. Boxing could happen on the boundary with JS as part of the JS API.

I understand this is incompatible with the current design of type imports, but have alternative designs that would allow this been considered? It seems sufficiently simpler for the abstract type provider to be able to just use an i32 that it would be great to see discussion of why disallowing that is beneficial.

[RossTate]

@tlively, just to separate the concern of abstraction from that of separate compilation, would it be alright if we adjusted your question slightly to use i31ref (or some similar numbers-compatible-with-references-type), instead of i32?

[tlively]

My question is fundamentally about importing abstract types known to be represented as i32s and exporting i32s as an abstract type, so modifying it that way would change its character. I'm not thinking about separate compilation, so I don't see what the connection is. My motivation is that it would be simpler for tooling if abstract types did not depend on additional primitive types.

[RossTate]

Ah, got it. I have been investigating that option. The only problem I have encountered so far is separate compilation. I believe that problem is surmountable through a variety of means, but before going further into that it would be useful to first hear from @rossberg if he knows of some other factor involved.

[rossberg]

The connection to separate compilation is that the AOT compiler can't know yet if it's represented by an i32. Or if it does, it's not a very flexible import, e.g., it could not be instantiated with a reference, such as externref, which breaks primary use cases like WASI.

A compiler will generally need to know both the size of a type and whether or not it is a (tagged) pointer, e.g., to create relevant data structures for a GC if present (memory/stack frame layout etc). If you wanted to be able to abstract over some of that as well, then a number of codegen aspects would have to be deferred, specialised dynamically, and/or handled through more internal indirections and hidden arguments. Complexity of such a compilation model aside, it is not a given that it would improve performance overall.

Surely worth investigating at some point, but not something I would dare overloading the MVP with.

And my gut feeling is that it could well complicate tooling as well. :)

[RossTate]

Okay, so if separate compilation is the only reason, then one thing I've wondered is if we could solve the problem by extending compile in the JS API to take an additional type_imports parameter. This additional type_imports parameter could be the actual types themselves, or maybe it could just be a rough description of the types, e.g. one of num32/num64/ref. There are details and design choices that need to be considered, but does that general strategy seem like an option worth exploring?

[rossberg]

Closing via #8.