ComWrapper RCW Design question

[weltkante]

I've been told by @AaronRobinsonMSFT that the ComWrapper API will be the future of COM interop development. I'm interested in this development as we make heavy use of COM interop in our products, both for win32 interop and for communication between our own components, including across languages and across processes.

ComWrapper is brand new and I'm having a bit trouble following up, CCW generation seems to have some good examples but RCW less so. From the test code and what I can gather from the issues I have doubts whether the RCW API Design of ComWrapper will hold up for general purpose use.

From what I've been linked to it looks like you are supposed to override CreateObject and construct a managed wrapper around the vtable.

As far as I'm reading out of various issue discussions (e.g. here) the design has been validated for WinUI/WinRT usage who have IInspectable to allow determining what the vtable represents. I'm worried this design of just passing a vtbl pointer along will not work for classic COM or other COM-like APIs which don't make use of IInspectable and have an unknown set of interfaces.

I've read the suggestion to "just query for all IIDs you know" to figure out what the object supports - this is a no-go for me as the approach doesn't scale (some of our libraries can support on the magnitude of hundreds of COM interfaces imported from TLBs, I cannot imagine that querying for all of them on each RCW would scale)

In old CLR COM I understand the RCW is an object that I can cast to an interface and it will on-demand map that vtable to an implementation of that interface. What I would have expected to see for an API to replace RCW is that I can build up one interface at a time lazily as C# performs the casts.

Am I missing something here? If the lazy cast pattern cannot be built into ComWrapper directly I can also imagine implementing a magic interface like ICustomQueryInterface on the RCW, just for managed casts. The current RCWs have to be solving it somehow so it can't be technically impossible to generalize the concept if it wasn't part of the current design.

[weltkante]

Had a vague memory that something like ICustomQueryInterface existed for managed casts, found ICastable but it doesn't seem to be available to user code. Is this what RCWs use? Would it make sense to follow up on #23727 to make ICastable usable for custom RCWs with ComWrapper? Or is it more appropriate to replace it with something more modern?

[AaronRobinsonMSFT]

@weltkante Thanks for filing this issue.

I want to clarify the following statement:

the ComWrapper API will be the future of COM interop development

The above shouldn't be read to imply that we will be removing the existing COM interop. The built-in COM interop will exist for the foreseeable future - which can probably be read as "forever" in this instance. It also doesn't mean that we will be avoiding fixing bugs or opting out of new features to make the system better.

What it does mean is that the existing built-in COM interop system is the stable and reliable path for COM and we will continue to make decisions based on the fact. It is meant to be stable for existing users to continue to port code from .NET Framework to .NET Core and with that it does mean certain issues may not get fixed in the spirit of ensuring compatibility. Time may change that but for now that is the case.

I've read the suggestion to "just query for all IIDs you know" to figure out what the object supports - this is a no-go for me as the approach doesn't scale (some of our libraries can support on the magnitude of hundreds of COM interfaces imported from TLBs, I cannot imagine that querying for all of them on each RCW would scale)

Yep. That is the current suggestion and it is correct arbitrary QI for the world will not scale. It is indeed designed with the added support for IInspectable which helps immensely. That is what makes ComWrappers possible to support the WinRT experience immediately.

In old CLR COM I understand the RCW is an object that I can cast to an interface and it will on-demand map that vtable to an implementation of that interface. What I would have expected to see for an API to replace RCW is that I can build up one interface at a time lazily as C# performs the casts.

That is how the existing built-in system works. Most vtables are generated on the fly and this does provide the laziness that permits on-demand query of interfaces and the magic of support. A draw back of this approach is it is not AOT friendly, which is a focus for .NET Core going forward. However, even with that focus this issue is still an important problem to solve.

With the current ComWrappers system the exact same lazy generation of vtables could be created through a type of aggregation - identical to how the built-in system works. As mentioned elsewhere it isn't a small amount of code - in fact it is a substantial amount of code with many many many edge cases. The point though is that it is possible to provide this support entirely in managed code using the ComWrappers API as a basic building block. With this simple building block it is possible for the runtime or some community member to create an entire COM interop solution external to the runtime. As a consequence of that is the ability to change behavior and fix bugs and have users decide when to accept the changes or stay on their existing COM interop solution.

This API represents the first part of a larger story to enable the community as well as the .NET Core platform to experiment with different approaches to addressing real customer needs with COM interop and have it evolve.

[AaronRobinsonMSFT]

/cc @jkoritzinsky @elinor-fung @jeffschwMSFT

[AaronRobinsonMSFT]

... found ICastable but it doesn't seem to be available to user code.
Or is it more appropriate to replace it with something more modern?

All good questions. ICastable is an option or as mentioned something newer. ICastable has some unusual semantics but there is a better/safer one. I can't recall what it is called. I believe @davidwrighton is familiar with the ICastable replacement.

ICastable or something similar is the next building block for the COM interop story.

[jkoritzinsky]

The safer option is ICastableObject, which we would have to port from CoreRT over to CoreCLR.

[davidwrighton]

Indeed. ICastableObject would allow arbitrary interface dispatch. However, the implementation is quite fiddly to test and implement, and I don't believe we have resources to implement it for .NET 5.

[AaronRobinsonMSFT]

I don't believe we have resources to implement it for .NET 5.

Yes that is important. None of this would be for .NET 5.

[weltkante]

None of this would be for .NET 5.

What does "none of this" refer to, the discussion in this issue, or is the ComWrapper story itself currently being developed for post-5.0?

[AaronRobinsonMSFT]

Sorry, any kind of ICastable solution is post-.NET 5. The ComWrappers itself is in for .NET 5.0 to enable our WinRT story.

[AaronRobinsonMSFT]

@weltkante Looks like we are going to explore the ICastableObject option in this release. I am going to close this issue and defer to the official proposal - #36654. Let's continue the conversation there.