Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Consider adding {ReadOnly}SpanMemoryStream types #22838

Open
stephentoub opened this issue Jul 18, 2017 · 2 comments
Open

Consider adding {ReadOnly}SpanMemoryStream types #22838

stephentoub opened this issue Jul 18, 2017 · 2 comments
Labels
api-needs-work API needs work before it is approved, it is NOT ready for implementation area-System.Memory tenet-performance Performance related issue
Milestone

Comments

@stephentoub
Copy link
Member

stephentoub commented Jul 18, 2017

Just as we have a MemoryStream that works with byte[] and an UnmanagedMemoryStream that works with a byte* and a length, we should support treating Memory<byte> and ReadOnlyMemory<byte> as streams.

It’s possible we could get away with reimplementing MemoryStream on top of Memory<byte>, but more than likely this would introduce regressions for at least some existing use cases, and it doesn't work with ReadOnlyMemory<byte>.

It would be nice if we could potentially write something ala:

Stream s = memory.TryGetArray(out var segment) ?
    new MemoryStream(segment.Array, segment.Offset, segment.Count) :
    new UnmanagedMemoryStream(&memory.DangerousGetPinnableReference(), buffer.Length);

but it's possible that TryGetArray could return false but the wrapped memory not inherently pinned, in which case the UnmanagedMemoryStream case would be wrapping a pointer to memory that could get moved. To pin it and keep track of the pinning, you'd need a wrapper Stream type anyway.

So, we could have two new stream types for these specific types:

namespace System.IO
{
    public class BufferStream : Stream
    {
        public BufferStream(Memory<byte> buffer);
        public BufferStream(Memory<byte> buffer, bool publiclyVisible);
        public bool TryGetBuffer(out Memory<byte> buffer);// overrides of all members on Stream
    }

    public class ReadOnlyBufferStream : Stream
    {
        public ReadOnlyBufferStream(ReadOnlyMemory<byte> buffer);
        public ReadOnlyBufferStream(ReadOnlyMemory<byte> buffer, bool publiclyVisible);
        public bool TryGetBuffer(out ReadOnlyMemory<byte> buffer);// overrides of all members on Stream, with those that write throwing NotSupportedException
    }
}

The name of BufferStream is unfortunately close to that of BufferedStream, and they mean very different things, but I’m not sure that’s important enough to consider a less meaningful name.

Alternatively, we could hide these behind a factory, e.g.

public static Stream ToStream(this Memory<byte> buffer) => new BufferStream(buffer);
public static Stream ToStream(this ReadOnlyMemory<byte> buffer) => new ReadOnlyBufferStream(buffer);

to keep them from needing to be public.

Especially if we do the latter, we would want to consider either adding TryGetBuffer virtuals to the base Stream class, or introducing an interface that could be queried for and which exposing such methods.
Then for example code that uses Streams and has optimizations when working directly with the underlying data can query for the interface and special-case when the underlying Buffer<T> can be accessed, e.g.

namespace System
{
    public interface IHasBuffer<T>
    {
        bool TryGetBuffer(out Memory<T> buffer);
        bool TryGetBuffer(out ReadOnlyMemory<T> buffer);
    }
}

We could implement this not only on BufferStream and ReadOnlyBufferStream, but also on MemoryStream, UnmanagedMemoryStream, and even on non-streams, basically anything that can hand out a representation of its internals as buffers.

@KrzysztofCwalina
Copy link
Member

KrzysztofCwalina commented Sep 13, 2017

Now that we renamed the type to Memory, the problem with BufferStream name goes away. Unfortunately another problem is created :-), i.e. what would we now call this stream type?

Also, where do we expect it to live? System.Memory.dll?

cc: @ahsonkhan

@ohsorry
Copy link

ohsorry commented Feb 23, 2020

where do we expect it to live

As I proposed. Now read and write Memory<byte> can only use Slices/CopyTo and then track the position manually . It would be convenient if Memory<byte> could be treated like a stream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-needs-work API needs work before it is approved, it is NOT ready for implementation area-System.Memory tenet-performance Performance related issue
Projects
None yet
Development

No branches or pull requests

6 participants