Support "local" types which know how to convert to and from primitive types - eg, Guid, Url

[data-sync-user]

In app-services, we've had to compromise on the interfaces we expose, particularly around Guid and Url. Here's my latest take on this:

How about we "just" provide a way for consumers to provide a list of "external types" - a list of [(name, raw_type)]

• eg, ('Guid', String)

Doing this allows us to write in the .udl:

  Guid get_guid(optional Guid? s);

and then it's fairly easy to have:

1. the Rust side of the generation just calls fn get_guid(guid: Option<Guid>) -> Guid (assuming that type exists and implements the uniffi traits)
2. the "other" side of the generation world to just use strings - in Python, get_guid(None)) works and returns a string.

and you can imagine (2) being optionally smarter - eg, we can probably arrange for projects to also arrange for a suitable Guid type to be used in the bindings.

I haven't thought too much about how to spell these types, but for the sake of example, let's assume in uniffi.toml with something like:

[local_types]

[local_types.Guid]
raw = "String"
# Python gets special support, other languages get a string.
python = { lower = "some_package.GuidType.to_string", lift = "some_package.GuidType.from_string" }

[local_types.JSONObject]
raw = "String"
python = {...}
kotlin = {...}
# etc!

This would probably allow a consumer to fully implement #440 without any help from us and probably even backout timestamp/duration support.

(To be clear, I'm not saying we should exclude JSON support or back timestamps out, but we could)

FWIW, I've got my get_guid() example above working in a very very hacky way, including painfully hand-written implemention of ViaFfi for Guid

• but close enough to convince me it's doable.

WDYT?

┆Issue is synchronized with this Jira Task
┆Epic: UniFFI
┆Sprint End Date: 2021-06-25

[data-sync-user]

➤ Ryan Kelly commented:

One thing to be aware of here is that this can interfere with the use of generics in function signatures, like this example ( https://github.com/mozilla/uniffi-rs/blob/d58e92f58a48e65b0c6ec10c0cb22d103a266e50/examples/todolist/src/lib.rs#L53 ) from the TodoList crate:

fn add_item<S: Into>(&self, item: S) -> Result<()> {
...
} This is designed to make it easier to call add_item directly from Rust code passing any string-like thing, while also accepting an actual String as is currently passed by the generated UniFFI bindings. If the generated UniFFI bindings tried to use into() on a call to this method:

obj.additem(item.try_lift().into())Then Rust doesn't have enough information to figure out the concrete type of the argument, and will give a compile error.

Probably that's not a bad tradeoff in practice, and we're better off optimising for good foreign-language bindings than for making an API that's nice to call from Rust code. Just something to keep in mind.

Another issue which might mean this isn't desirable in the first place is that it causes a lack of symmetry between the
rust code and the foreign language code.

This seems to mesh with the "transform towers" idea that jhugman wrote up last week.

Alternatively, pushing on #416 might be better - we could probably deduce all the above.

Yep. This would do away with the need for [Self=ByArc] and [ByRef] as well :-)

[data-sync-user]

➤ Mark Hammond commented:

One thing to be aware of here is that this can interfere with the use of generics in function signatures,

Right! I guess we can have our cake and eat it too if we require an attribute even for the plain into()

[data-sync-user]

➤ Mark Hammond commented:

OTOH: leaning into the From and TryFrom traits smells wrong. They are commonly used traits and Ryan's example is just a taste of where this might go bad. Leaning into our own *Ffi traits might offer a better outcome? I'll try and restate the problem rather than the solution 😀

[data-sync-user]

➤ Mark Hammond commented:

FWIW, I've edited the first comment to describe a hand-wavey proposal which I'd love thoughts on and hidden earlier comments)

[data-sync-user]

➤ Ryan Kelly commented:

So, I think this is headed in broadly a good direction, and it's a direction we should explore a bit in an iterative fashion. It seems to intersect really strongly with James' Transform Towers ( https://docs.google.com/document/d/1nibSjM9Vslqj6GKVxePq6fkOp4Yg4euzGeVidHBUnC4/ ) proposal.

As a first attempt, I feel like I'd rather see the types named explicitly in the interface definition somehow, rather than being squirreled away in a config file. Maybe we could use the typedef syntax ( https://heycam.github.io/webidl/#idl-typedefs ) as a starting point? The details of exactly how to codegen for them in a foreign language might still need to be in a config file, but it sounds like you're not necessarily interested in pushing on the foreign-language side right away, so much as allowing richer types on the Rust side.

I wonder if something like typedef [Extern] String Guid; in the .udl and a hand-written impl ViaFfi for Guid would be enough to take a first step here?

[data-sync-user]

➤ Ryan Kelly commented:

FWIW, I can imagine a similar annotation being used to access types from another component, which in theory would already come with their own ViaFfi implementation.

[data-sync-user]

➤ Mark Hammond commented:

FWIW, I can imagine a similar annotation being used to access types from another component

Yeah, I've been pondering this too, but then I started having images of uniffizords and had to seek therapy.

[data-sync-user]

➤ Ryan Kelly commented:

I started having images of uniffizords and had to seek therapy.

You are going to love the next Google Doc that I'm working on...