Consider implementing ByteArray and Array in Inko with a set of primitive instructions

[yorickpeterse]

ByteArray is currently implemented using a mixture of pure Inko code and built-in functions. Since byte arrays tend to get rather large, operations such as appending, slicing, etc may be too expensive when implemented in Inko (e.g. a series of push() calls). Adding all sorts of built-in functions also gets really annoying.

A possible solution is to add a set of primitive memory management instructions to the VM. These operate on regular unmanaged memory, and could be used to implement ByteArray (and possibly Array in the future) in Inko itself; instead of relying on an ever growing list of built-in functions. VM methods that currently produce a ByteArray would instead produce something like a *mut u8, while methods expecting a ByteArray would expect a &mut [u8] (created from a *mut u8 and a length).

The instructions would be at least as follows:

• alloc(size in bytes)
• free(pointer)
• copy(from, to, size in bytes)
• realloc(pointer, new size in bytes)
• write(pointer, byte)
• read(pointer)

To handle differently sized types we'd probably also have to pass a type size argument, instead of always operating on bytes.

Moving Array over to such a setup will probably be more difficult, as there are several places where the VM directly uses it. For example, the built-in function directory_list returns an Array.

Either way, this will require some thinking. It might not even be worth it if/when we move to a native code compiler, as at that point we'd probably be able to better optimise built-in function calls and just do the ugly work in Rust.

Depends on

• Type-safe C FFI #290: if we add the FFI first, we don't need to extend the runtime with any extra functions

[yorickpeterse]

Implementing ByteArray in Inko poses a problem: various runtime functions read data into a ByteArray, and the underlying Rust methods expect a resizable Vec<u8> in these cases. If we implemnt ByteArray in Inko entirely, we no longer have a Vec that we can rely upon. We can't use Vec::from_raw_parts either, as the allocator we use may be different from the one used by Vec (at least on paper, in practise they'll probably be the same).

This likely means we'll need to move more low-level plumbing into the standard library. For example, rt::socket::socket_output_slice probably needs to be moved to the standard library in some capacity, such that we can resize the buffer and then pass it (with enough space) to the raw socket operations.

As a starting point it's probably best to implement Array in Inko, but leave ByteArray as-is for the time being.