Memory management for WinRT Classes

[timsneath]

Originally posted by @halildurmus in #431 (comment)

Assume that we're using the Mixin model for generating WinRT classes.

There are some cases where we can't deallocate pointers in the generated code. Therefore, it is the consumers' responsibility to deallocate them once they're done using them.

These are:

• WinRT class pointers e.g. Calendar's pointer (these pointers are allocated by constructors such as Calendar's default constructor, factory constructors, methods, properties, or by consumers using ActivateObject or other helper methods)
• WinRT mixin pointers e.g. pointers of the ICalendar and ITimeZoneOnCalendar
• APIs that return IVector<T extends IInspectable> or IVectorView<T extends IInspectable> e.g. IVector<StorageFile>. As of now, these are exposed as Pointer<COMObject>. We only auto-generate APIs that return IVector<String> and IVectorView<String> at the moment but we will add support for these in the future. Here is the one I manually generated.

...and probably much more in the future as the WinRT projection matures (asynchronous APIs, IMap & IMapView...).

The problem is that there are many situations where the consumers need to free pointers manually and I think it is pretty common to forget freeing some of the pointers (especially if you're using a lot of them). Another thing that is quite painful to deal with is that if you accidentally call free method twice on the same pointer, the process will crash and good luck figuring out the source of the problem (I got bitten by this many times :( ).

I've been trying to find a way to fix these issues for the past few days and I have some ideas I'd like to share with you.

I'll start with the first design. We could consider introducing a class that manages the pointers that could not be freed in the generated code:

class WinRTAllocator {
  static final allocator = Arena();

  /// Releases all resources managed by the [allocator].
  static void freeAll() => allocator.releaseAll(reuse: true);
}

This class would contain a static allocator to be used by the generated code and a static method to free all pointers. We would expose this to the outside world so that the consumers would have a way of freeing all pointers by calling just a single method once they're done using WinRT APIs.

Here are the cases where we need to modify the generator to use this allocator:

• WinRT class pointers
• WinRT mixin pointers (We would pass WinRTAllocator.allocator to toInterface method as a parameter)
• APIs that return IVector<T extends IInspectable> or IVectorView<T extends IInspectable>

...and much more in the future...

This design is much simpler to use, but if consumers use WinRT APIs heavily, I'm worried that the memory usage may be higher until they call WinRTAllocator.freeAll() (there may be some cases where the consumer is using tightly coupled APIs and because of this they can't call freeAll method between calls and must wait for all calls to complete).

To solve this, here is the final design I came up with. First, we could create our own Arena implementation by slightly modifying dart:ffi's implementation to allow freeing specific pointers from the _managedMemoryPointers list (by modifying Arena's free method):

class WinRTAllocator {
  static final allocator = _Arena();

  /// Releases memory allocated on the native heap.
  static void free(Pointer<NativeType> pointer) => allocator.free(pointer);

  /// Releases all resources managed by the [allocator].
  static void freeAll() => allocator.releaseAll(reuse: true);
}

class _Arena implements Allocator {
  /// The [Allocator] used for allocation and freeing.
  final Allocator _wrappedAllocator;

  /// Native memory under management by this [Arena].
  final List<Pointer<NativeType>> _managedMemoryPointers = [];

  /// Callbacks for releasing native resources under management by this [Arena].
  final List<void Function()> _managedResourceReleaseCallbacks = [];

  bool _inUse = true;

  /// Creates a arena of allocations.
  ///
  /// The [allocator] is used to do the actual allocation and freeing of
  /// memory. It defaults to using [calloc].
  _Arena([Allocator allocator = calloc]) : _wrappedAllocator = allocator;

  /// Allocates memory and includes it in the arena.
  ///
  /// Uses the allocator provided to the [Arena] constructor to do the
  /// allocation.
  ///
  /// Throws an [ArgumentError] if the number of bytes or alignment cannot be
  /// satisfied.
  @override
  Pointer<T> allocate<T extends NativeType>(int byteCount, {int? alignment}) {
    _ensureInUse();
    final p = _wrappedAllocator.allocate<T>(byteCount, alignment: alignment);
    _managedMemoryPointers.add(p);
    return p;
  }

  /// Registers [resource] in this arena.
  ///
  /// Executes [releaseCallback] on [releaseAll].
  ///
  /// Returns [resource] again, to allow for easily inserting
  /// `arena.using(resource, ...)` where the resource is allocated.
  T using<T>(T resource, void Function(T) releaseCallback) {
    _ensureInUse();
    releaseCallback = Zone.current.bindUnaryCallback(releaseCallback);
    _managedResourceReleaseCallbacks.add(() => releaseCallback(resource));
    return resource;
  }

  /// Registers [releaseResourceCallback] to be executed on [releaseAll].
  void onReleaseAll(void Function() releaseResourceCallback) {
    _managedResourceReleaseCallbacks.add(releaseResourceCallback);
  }

  /// Releases all resources that this [Arena] manages.
  ///
  /// If [reuse] is `true`, the arena can be used again after resources
  /// have been released. If not, the default, then the [allocate]
  /// and [using] methods must not be called after a call to `releaseAll`.
  ///
  /// If any of the callbacks throw, [releaseAll] is interrupted, and should
  /// be started again.
  void releaseAll({bool reuse = false}) {
    if (!reuse) {
      _inUse = false;
    }
    // The code below is deliberately wirtten to allow allocations to happen
    // during `releaseAll(reuse:true)`. The arena will still be guaranteed
    // empty when the `releaseAll` call returns.
    while (_managedResourceReleaseCallbacks.isNotEmpty) {
      _managedResourceReleaseCallbacks.removeLast()();
    }
    for (final p in _managedMemoryPointers) {
      _wrappedAllocator.free(p);
    }
    _managedMemoryPointers.clear();
  }

  /// Releases memory allocated on the native heap.
  @override
  void free(Pointer<NativeType> pointer) {
    final isRemoved = _managedMemoryPointers.remove(pointer);
    if (isRemoved) _wrappedAllocator.free(pointer);
  }

  void _ensureInUse() {
    if (!_inUse) {
      throw StateError(
          'Arena no longer in use, `releaseAll(reuse: false)` was called.');
    }
  }
}

(I'd like to point out that calling WinRTAllocator's free multiple times on the same pointer won't crash your process as the implementation checks if the pointer exists in the _managedMemoryPointers list before calling the native API.)

Then, we could create a dispose method in IInspectable:

class IInspectable extends IUnknown {
  ...

  /// Releases all memory allocated by this object.
  void dispose() => WinRTAllocator.free(ptr);
}

...and modify WinRT classes and mixins like this:

// icalendar.dart
mixin ICalendar on IInspectable {
  late final ptrICalendar =
      toInterface(IID_ICalendar, allocator: WinRTAllocator.allocator);
  // ...
}

// itimezoneoncalendar.dart
mixin ITimeZoneOnCalendar on IInspectable {
  late final ptrITimeZoneOnCalendar =
      toInterface(IID_ITimeZoneOnCalendar, allocator: WinRTAllocator.allocator);
  // ...
}

// calendar.dart
class Calendar extends IInspectable with ICalendar, ITimeZoneOnCalendar {
  Calendar() : super(ActivateWinRTClass(_className));
  Calendar.fromPointer(super.ptr);

  static const _className = 'Windows.Globalization.Calendar';

  // ...

  @override
  void dispose() {
    WinRTAllocator.free(ptr);
    WinRTAllocator.free(ptrICalendar);
    WinRTAllocator.free(ptrITimeZoneOnCalendar);
  }
}

With this design, consumers have the flexibility to use the dispose method of the WinRT classes in combination with the freeAll method of the WinRTAllocator for memory management. They can decide which option is more suitable for their needs or preferences. Some might prefer to just use freeAll for its simplicity while others may use both of them to make their applications more efficient.

[timsneath]

So dart:ffi is in the process of adding support for finalizers:
dart-lang/sdk#47772

I don't fully understand the design yet, which is why I was slow to replying -- but I think this will mostly mitigate the challenges here. We'd have to implement an interface to tell Dart how to dispose of COM / WinRT objects, but the free call will be made automatically when the object goes out of scope (and there will be a consistent way to call this manually).

While the core Finalizer mechanism is now available in Dart 2.17, we need the final checkbox from the issue above to be completed in order that we can use it from Dart code. Here's the design doc:
https://github.com/dart-lang/language/blob/master/accepted/future-releases/1847%20-%20FinalizationRegistry/proposal.md#isolate-independent-native-functions

[halildurmus]

Thanks, that's great! This will make using WinRT APIs much easier for consumers.

[timsneath]

Does it make sense for this work to be completed first ({cough} @dcharkes 🤣) , and then take another look at our memory management strategy?

[dcharkes]

Yes! 😄

I'm making progress on a prototype that will enable us to use NativeFinalizer with HeapFree without compiling C code separately. Here is a sneak peak. The magic is in Pointer.isolateIndependentFunction<IntPtr Function(Pointer)>(winFree) which essentially makes the Dart compiler compile a bit of C code which calls GetProcessHeap to pass in to HeapFree along with the pointer. That "compiled C code" can then be passed to the NativeFinalizer as callback.

Unfortunately, I cannot give any guarantees when this will be fully supported. (Or if the current implementation approach is the right one.) In other words: We're working on it. 🤞

[halildurmus]

Does it make sense for this work to be completed first ({cough} @dcharkes 🤣) , and then take another look at our memory management strategy?

Yeah, it makes sense :)

[timsneath]

Just a question, @dcharkes -- have you looked at CoTaskMemAlloc and CoTaskMemFree? As I understand it, they share the same signature as malloc() and free(). If we used those signatures, would we be able to use the existing NativeFinalizer?

While they're designed with OLE/COM in mind, they are generally applicable. I've seen one reference to suggest that there's a slight overhead with these calls, but I can imagine it might be a rounding error for the kind of usage we're talking about here.

[dcharkes]

I have not. That might just be the perfect solution!

The issue with malloc and free on Windows was that if you have two dlls and you allocate in with the first dll and then try to free with the second that things go haywire.

When your module calls new or malloc, the memory can only be freed by your module calling delete or free. If another module calls delete or free, that will use the C/C++ runtime of that other module which is not the same as yours.

https://devblogs.microsoft.com/oldnewthing/20060915-04/?p=29723

One option is to use a fixed external allocator such as LocalAlloc or CoTaskMemAlloc. These are allocators that are universally available and don’t depend on which version of the compiler you’re using.

@timsneath can you check that if you change package:ffi locally (with a path dependency in win32 to your local package:ffi) to use CoTaskMemAlloc and CoTaskMemFree all your win32 tests succeed? (And feel free to make a PR afterwards if it works 😃 )

And yes, then you could use NativeFinalizer! 🚀

[timsneath]

I'll give it a try!

[dcharkes]

I've released package:ffi 2.0.0 - happy memory managing! 😀

[halildurmus]

Great, thanks!