From 7ede7b7cac4bda787af39859fd228f197c11bf4c Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 30 Jun 2020 13:56:28 -0700 Subject: [PATCH 01/26] update terminology to lending stream, rather than detached stream Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 32 ++++++++++++++++++-------------- 1 file changed, 18 insertions(+), 14 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 84d1981b..7265c126 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -289,25 +289,29 @@ for elem in stream { ... } Designing this extension is out of scope for this RFC. However, it could be prototyped using procedural macros today. -## "Attached" streams +## "Lending" streams -There has been much discussion around attached/detached streams. +There has been much discussion around lending streams (also referred to as attached streams). ### Definitions [Source](https://smallcultfollowing.com/babysteps/blog/2019/12/10/async-interview-2-cramertj-part-2/#the-need-for-streaming-streams-and-iterators) -In a **detached** stream, the `Item` that gets returned by `Stream` is "detached" from self. This means it can be stored and moved about independently from `self`. -In an **attached** stream, the `Item` that gets returned by `Stream` may be borrowed from `self`. It can only be used as long as the `self` reference remains live. +In an **lending** stream (also known as an "attached" stream), the `Item` that gets returned by `Stream` may be borrowed from `self`. It can only be used as long as the `self` reference remains live. + +In a **non-lending** stream (also known as a "detached" stream), the `Item` that gets returned by `Stream` is "detached" from self. This means it can be stored and moved about independently from `self`. + +This RFC does not cover the addition of lending streams (streams as implemented through this RFC are all non-lending streams). -This RFC does not cover the addition of attached/detached owned/borrowed streams. We can add the `Stream` trait to the standard library now and delay -adding in this distinction between two types of streams. The advantage of this -is it would allow us to copy the `Stream` trait from `futures` largely 'as is'. -The disadvantage of this is functions that consume streams would first be written -to work with `Stream`, and then potentially have to be rewritten later to work with -`AttachedStream`s. +adding in this distinction between the two types of streams - lending and +non-lending. The advantage of this is it would allow us to copy the `Stream` +trait from `futures` largely 'as is'. + +The disadvantage of this is functions that consume streams would +first be written to work with `Stream`, and then potentially have +to be rewritten later to work with `LendingStream`s. ### Current Stream Trait @@ -327,10 +331,10 @@ pub trait Stream { This trait, like `Iterator`, always gives ownership of each item back to its caller. This offers flexibility - such as the ability to spawn off futures processing each item in parallel. -### Potential Attached Stream Trait +### Potential Lending Stream Trait ```rust -impl AttachedStream for S +impl LendingStream for S where S: Stream, { @@ -346,10 +350,10 @@ where ``` This is a "conversion" trait such that anything which implements `Stream` can also implement -`Attached Stream`. +`Lending Stream`. This trait captures the case we re-use internal buffers. This would be less flexible for -consumers, but potentially more efficient. Types could implement the `AttachedStream` +consumers, but potentially more efficient. Types could implement the `LendingStream` where they need to re-use an internal buffer and `Stream` if they do not. There is room for both. We would also need to pursue the same design for iterators - whether through adding two traits From 3ccfcb06f3cc5d89b1ba31e249716177569d1ba3 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 30 Jun 2020 14:56:35 -0700 Subject: [PATCH 02/26] adds more information on IntoStream trait Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 58 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 57 insertions(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 7265c126..f424d542 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -235,17 +235,73 @@ Designing such a migration feature is out of scope for this RFC. ### IntoStream +**Iterators** + Iterators have an `IntoIterator` that is used with `for` loops to convert items of other types to an iterator. +```rust +pub trait IntoIterator where + ::Item == Self::Item, +{ + type Item; + + type IntoIter: Iterator; + + fn into_iter(self) -> Self::IntoIter; +} +``` + +Examples taken from the Rust docs on [for loops and into_iter]](https://doc.rust-lang.org/std/iter/index.html#for-loops-and-intoiterator) + * `for x in iter` uses `impl IntoIterator for T` + +```rust +let values = vec![1, 2, 3, 4, 5]; + +for x in values { + println!("{}", x); +} +``` + +Desugars to: + +```rust +let values = vec![1, 2, 3, 4, 5]; +{ + let result = match IntoIterator::into_iter(values) { + mut iter => loop { + let next; + match iter.next() { + Some(val) => next = val, + None => break, + }; + let x = next; + let () = { println!("{}", x); }; + }, + }; + result +} +``` * `for x in &iter` uses `impl IntoIterator for &T` * `for x in &mut iter` uses `impl IntoIterator for &mut T` +**Streams** + We may want a trait similar to this for `Stream`. The `IntoStream` trait would provide a way to convert something into a `Stream`. This trait could look like this: -[TO BE ADDED] +```rust +pub trait IntoStream where + ::Item == Self::Item, +{ + type Item; + + type IntoStream: Stream; + + fn into_stream(self) -> Self::IntoStream; +} +``` ### FromStream From b5f7031b410c1907dd8b8d09ebf4ab25b1cb06ff Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 30 Jun 2020 15:10:42 -0700 Subject: [PATCH 03/26] expands on FromStream trait Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 56 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 55 insertions(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index f424d542..ff755c60 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -305,13 +305,67 @@ pub trait IntoStream where ### FromStream +**Iterators** + Iterators have an `FromIterator` that is used to convert iterators into another type. +```rust +pub trait FromIterator { + + fn from_iter(iter: T) -> Self + where + T: IntoIterator; +} +``` + +It should be noted that this trait is rarely used directly, instead used through Iterator's collect method ([source](https://doc.rust-lang.org/std/iter/trait.FromIterator.html)). + +```rust +pub trait Interator { + fn collect(self) -> B + where + B: FromIterator, + { ... } +} +``` + +Examples taken from the Rust docs on [iter and collect]](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.collect) + + +```rust +let a = [1, 2, 3]; + +let doubled: Vec = a.iter() + .map(|&x| x * 2) + .collect(); + +``` + +**Streams** + We may want a trait similar to this for `Stream`. The `FromStream` trait would provide way to convert a `Stream` into another type. This trait could look like this: -[TO BE ADDED] +```rust +pub trait FromStream { + + fn from_stream(iter: T) -> Self + where + T: IntoStream; +} +``` + +We could potentially include a collect method for Stream as well. + +```rust +pub trait Stream { + fn collect(self) -> B + where + B: FromStream, + { ... } +} +``` ## Other Traits From f6cd705c2592f6bfc49fd262125900fa0829c77f Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 30 Jun 2020 15:21:41 -0700 Subject: [PATCH 04/26] add more info on converting between lending streams and non-lending streams Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index ff755c60..0da259c5 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -470,8 +470,12 @@ We would also need to pursue the same design for iterators - whether through add or one new trait with a "conversion" from the old trait. This also brings up the question of whether we should allow conversion in the opposite way - if -every "Detached" stream can become an attached one, should _some_ detached streams be able to -become attached ones? These use cases need more thought, which is part of the reason +every non-lending stream can become a lending one, should _some_ non-lending streams be able to +become lending ones? + +We can say that, as the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to the [coherence conflict](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a667a7560f8dc97ab82a780e27dfc9eb) shown here. + +These use cases need more thought, which is part of the reason it is out of the scope of this particular RFC. ## Generator syntax From a6a69a03d9809d3f5ade1220520622c63a2b504b Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 30 Jun 2020 15:28:45 -0700 Subject: [PATCH 05/26] even more information about converting Streams to LendingStreams Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 25 ++++++++++++++++++++++--- 1 file changed, 22 insertions(+), 3 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 0da259c5..0701c07e 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -473,10 +473,29 @@ This also brings up the question of whether we should allow conversion in the op every non-lending stream can become a lending one, should _some_ non-lending streams be able to become lending ones? -We can say that, as the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to the [coherence conflict](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a667a7560f8dc97ab82a780e27dfc9eb) shown here. +We can say that, as the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to a coherence conflict. -These use cases need more thought, which is part of the reason -it is out of the scope of this particular RFC. +If you have other impls like: + +```rust +impl Stream for Box where T: Stream +``` + +and + +```rust +impl LendingStream for Box where T: LendingStream +``` + +There is a coherence conflict for `Box`, so presumably it will fail the coherence rules. + +[More examples are available here](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=a667a7560f8dc97ab82a780e27dfc9eb). + +Resolving this would require either an explicit “wrapper” step or else some form of language extension. + +It should be noted that the same applies to Iterator, it is not unique to Stream. + +These use cases for lending/non-lending streams need more thought, which is part of the reason it is out of the scope of this particular RFC. ## Generator syntax [generator syntax]: #generator-syntax From 81a2d511cbae148fbf9ce793d53888e7283fc00f Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Thu, 2 Jul 2020 11:17:36 -0700 Subject: [PATCH 06/26] adds in information about the next method Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 46 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 0701c07e..e6687e1b 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -124,6 +124,52 @@ where } ``` +## Next method/struct + +We should also implement a next method, similar to [the implementation in the futures crate](https://docs.rs/futures-util/0.3.5/src/futures_util/stream/stream/next.rs.html#10-12). + +```rust +/// Future for the [`next`](super::StreamExt::next) method. +#[derive(Debug)] +#[must_use = "futures do nothing unless you `.await` or poll them"] +pub struct Next<'a, St: ?Sized> { + stream: &'a mut St, +} + +impl Unpin for Next<'_, St> {} + +impl<'a, St: ?Sized + Stream + Unpin> Next<'a, St> { + pub(super) fn new(stream: &'a mut St) -> Self { + Next { stream } + } +} + +impl FusedFuture for Next<'_, St> { + fn is_terminated(&self) -> bool { + self.stream.is_terminated() + } +} + +impl Future for Next<'_, St> { + type Output = Option; + + fn poll( + mut self: Pin<&mut Self>, + cx: &mut Context<'_>, + ) -> Poll { + self.stream.poll_next_unpin(cx) + } +} +``` + +This would allow a user to await on a future: + +```rust +while let Some(v) = stream.next().await { + +} +``` + # Reference-level explanation [reference-level-explanation]: #reference-level-explanation From 7c95855e0e3aaa875dbcb657a300d6928348f95f Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 7 Jul 2020 10:53:56 -0700 Subject: [PATCH 07/26] Update rfc-drafts/stream.md Co-authored-by: Yoshua Wuyts --- rfc-drafts/stream.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index e6687e1b..59d1e34a 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -129,7 +129,13 @@ where We should also implement a next method, similar to [the implementation in the futures crate](https://docs.rs/futures-util/0.3.5/src/futures_util/stream/stream/next.rs.html#10-12). ```rust -/// Future for the [`next`](super::StreamExt::next) method. +/// A future that advances the stream and returns the next value. +/// +/// This `struct` is created by the [`next`] method on [`Stream`]. See its +/// documentation for more. +/// +/// [`next`]: trait.Stream.html#method.next +/// [`Stream`]: trait.Stream.html #[derive(Debug)] #[must_use = "futures do nothing unless you `.await` or poll them"] pub struct Next<'a, St: ?Sized> { From 9f6aa055eed994763ba433172f772e412521c2d8 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 7 Jul 2020 10:54:30 -0700 Subject: [PATCH 08/26] Update rfc-drafts/stream.md Co-authored-by: Yoshua Wuyts --- rfc-drafts/stream.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 59d1e34a..155a6365 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -138,8 +138,8 @@ We should also implement a next method, similar to [the implementation in the fu /// [`Stream`]: trait.Stream.html #[derive(Debug)] #[must_use = "futures do nothing unless you `.await` or poll them"] -pub struct Next<'a, St: ?Sized> { - stream: &'a mut St, +pub struct Next<'a, S: ?Sized> { + stream: &'a mut S, } impl Unpin for Next<'_, St> {} From 1750343299d372df0a8fac1468e6896aa65b7e13 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 7 Jul 2020 10:54:44 -0700 Subject: [PATCH 09/26] Update rfc-drafts/stream.md Co-authored-by: Yoshua Wuyts --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 155a6365..316ff3af 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -163,7 +163,7 @@ impl Future for Next<'_, St> { mut self: Pin<&mut Self>, cx: &mut Context<'_>, ) -> Poll { - self.stream.poll_next_unpin(cx) + Pin::new(&mut *self.stream).poll_next(cx) } } ``` From 0050a3edf5587889bf55dec9af31abc4f88e9caa Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 7 Jul 2020 10:55:01 -0700 Subject: [PATCH 10/26] Update rfc-drafts/stream.md Co-authored-by: Yoshua Wuyts --- rfc-drafts/stream.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 316ff3af..f78b1c80 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -150,12 +150,6 @@ impl<'a, St: ?Sized + Stream + Unpin> Next<'a, St> { } } -impl FusedFuture for Next<'_, St> { - fn is_terminated(&self) -> bool { - self.stream.is_terminated() - } -} - impl Future for Next<'_, St> { type Output = Option; From 9d48ced32a1b3cc35f61952fe16441a84485018e Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 7 Jul 2020 10:55:39 -0700 Subject: [PATCH 11/26] Update rfc-drafts/stream.md Co-authored-by: Yoshua Wuyts --- rfc-drafts/stream.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index f78b1c80..62c06ff9 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -395,8 +395,7 @@ This trait could look like this: ```rust pub trait FromStream { - - fn from_stream(iter: T) -> Self + async fn from_stream(stream: T) -> Self where T: IntoStream; } From 8d192619e198e3a0e08d1618ef70aa576d41266c Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 7 Jul 2020 10:55:49 -0700 Subject: [PATCH 12/26] Update rfc-drafts/stream.md Co-authored-by: Yoshua Wuyts --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 62c06ff9..e8f1b296 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -405,7 +405,7 @@ We could potentially include a collect method for Stream as well. ```rust pub trait Stream { - fn collect(self) -> B + async fn collect(self) -> B where B: FromStream, { ... } From 06205c406fc768cbee4c9a727bf328a0f478925a Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 7 Jul 2020 09:54:08 -0700 Subject: [PATCH 13/26] adds in info on IntoStream Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index e8f1b296..6961565a 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -349,6 +349,26 @@ pub trait IntoStream where } ``` +This trait (as expressed by @taiki-e in [a comment on a draft of this RFC](https://github.com/rust-lang/wg-async-foundations/pull/15/files#r449880986)) makes it easy to write streams in combination with [async stream](https://github.com/taiki-e/futures-async-stream). + +```rust +type S(usize); + +impl IntoStream for S { + type Item = usize; + type IntoStream: impl Stream; + + fn into_stream(self) -> Self::IntoStream { + #[stream] + async move { + for i in 0..self.0 { + yield i; + } + } + } +} +``` + ### FromStream **Iterators** From d646a7e9d10367cec4f353357ede277724e8df03 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Tue, 7 Jul 2020 13:29:03 -0700 Subject: [PATCH 14/26] adds in more info about generators Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 6961565a..959c4769 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -575,9 +575,29 @@ or "owned" stream, the generator yield could return things that you own or things that were borrowed from your caller. ```rust -gen async fn foo() -> X { +gen async fn foo() -> Value { yield value; } ``` -Designing generator functions is out of the scope of this RFC. +After desugaring, this would result in a function like: + +```rust +fn foo() -> impl Iterator +``` + +```rust +async gen fn foo() -> Value +``` + +After desugaring would result in a function like: + +```rust +fn foo() -> impl Stream +``` + +If we introduce `-> Stream` first, we will have to permit `LendingStream` in the future. +Additionally, if we introduce `LendingStream` later, we'll have to figure out how +to convert a `LendingStream` into a `Stream` seamlessly. + +Further designing generator functions is out of the scope of this RFC. From c0bbd43c0ad4a82976486ca58f7bdb71ff188b30 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Thu, 9 Jul 2020 13:14:30 -0700 Subject: [PATCH 15/26] Update rfc-drafts/stream.md Co-authored-by: Tyler Mandry --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 959c4769..d9a4bb63 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -297,7 +297,7 @@ pub trait IntoIterator where } ``` -Examples taken from the Rust docs on [for loops and into_iter]](https://doc.rust-lang.org/std/iter/index.html#for-loops-and-intoiterator) +Examples taken from the Rust docs on [for loops and into_iter](https://doc.rust-lang.org/std/iter/index.html#for-loops-and-intoiterator) * `for x in iter` uses `impl IntoIterator for T` From 8cf7b37def5a2ab2d97844330b3bb6b22929ea48 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Thu, 9 Jul 2020 13:14:38 -0700 Subject: [PATCH 16/26] Update rfc-drafts/stream.md Co-authored-by: Tyler Mandry --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index d9a4bb63..ab7fc4e8 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -349,7 +349,7 @@ pub trait IntoStream where } ``` -This trait (as expressed by @taiki-e in [a comment on a draft of this RFC](https://github.com/rust-lang/wg-async-foundations/pull/15/files#r449880986)) makes it easy to write streams in combination with [async stream](https://github.com/taiki-e/futures-async-stream). +This trait (as expressed by @taiki-e in [a comment on a draft of this RFC](https://github.com/rust-lang/wg-async-foundations/pull/15/files#r449880986)) makes it easy to write streams in combination with [async stream](https://github.com/taiki-e/futures-async-stream). For example: ```rust type S(usize); From df39f127bfb8a292f344416bc6a5952ff78feec6 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Thu, 9 Jul 2020 13:15:59 -0700 Subject: [PATCH 17/26] Update rfc-drafts/stream.md Co-authored-by: Tyler Mandry --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index ab7fc4e8..30b9c3e5 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -395,7 +395,7 @@ pub trait Interator { } ``` -Examples taken from the Rust docs on [iter and collect]](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.collect) +Examples taken from the Rust docs on [iter and collect](https://doc.rust-lang.org/std/iter/trait.Iterator.html#method.collect) ```rust From a13dd486b28996605be590d40c1def5c454fcaee Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Thu, 9 Jul 2020 13:16:33 -0700 Subject: [PATCH 18/26] Update rfc-drafts/stream.md Co-authored-by: Tyler Mandry --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 30b9c3e5..f696b22a 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -473,7 +473,7 @@ There has been much discussion around lending streams (also referred to as attac [Source](https://smallcultfollowing.com/babysteps/blog/2019/12/10/async-interview-2-cramertj-part-2/#the-need-for-streaming-streams-and-iterators) -In an **lending** stream (also known as an "attached" stream), the `Item` that gets returned by `Stream` may be borrowed from `self`. It can only be used as long as the `self` reference remains live. +In a **lending** stream (also known as an "attached" stream), the `Item` that gets returned by `Stream` may be borrowed from `self`. It can only be used as long as the `self` reference remains live. In a **non-lending** stream (also known as a "detached" stream), the `Item` that gets returned by `Stream` is "detached" from self. This means it can be stored and moved about independently from `self`. From e39867985345d0e8c9a3ade7a9ef0f1bfda3ed62 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Thu, 9 Jul 2020 13:16:50 -0700 Subject: [PATCH 19/26] Update rfc-drafts/stream.md Co-authored-by: Tyler Mandry --- rfc-drafts/stream.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index f696b22a..719f8e3a 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -535,8 +535,8 @@ We would also need to pursue the same design for iterators - whether through add or one new trait with a "conversion" from the old trait. This also brings up the question of whether we should allow conversion in the opposite way - if -every non-lending stream can become a lending one, should _some_ non-lending streams be able to -become lending ones? +every non-lending stream can become a lending one, should _some_ lending streams be able to +become non-lending ones? We can say that, as the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to a coherence conflict. From d41c12babd39a1c1541342245b9b9c906f7750f7 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Thu, 9 Jul 2020 13:17:02 -0700 Subject: [PATCH 20/26] Update rfc-drafts/stream.md Co-authored-by: Tyler Mandry --- rfc-drafts/stream.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 719f8e3a..df37873a 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -538,7 +538,9 @@ This also brings up the question of whether we should allow conversion in the op every non-lending stream can become a lending one, should _some_ lending streams be able to become non-lending ones? -We can say that, as the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to a coherence conflict. +**Coherence** + +The impl above has a problem. As the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to a coherence conflict. If you have other impls like: From ddec9e0d79963c17de4a43dbe2c54d866456b742 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Thu, 9 Jul 2020 09:41:59 -0700 Subject: [PATCH 21/26] a few clarifications Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index df37873a..1ea57325 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -63,14 +63,20 @@ the current task to be re-awoken when the data is ready. ```rust // Defined in std::stream module pub trait Stream { + // Core items: type Item; - fn poll_next(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll>; + // Optional optimization hint, just like with iterators: #[inline] fn size_hint(&self) -> (usize, Option) { (0, None) } + + // Convenience methods (covered later on in the RFC): + fn next(&mut self) -> Next<'_, Self> + where + Self: Unpin; } ``` @@ -128,6 +134,10 @@ where We should also implement a next method, similar to [the implementation in the futures crate](https://docs.rs/futures-util/0.3.5/src/futures_util/stream/stream/next.rs.html#10-12). +In general, we have purposefully kept the core trait definition minimal. There are a number of useful extension methods that are available, for example, in the futures-stream crate, but we have not included them because they involve closure arguments, and we have not yet finalized the design of async closures. + +However, the core methods alone are extremely unergonomic. You can't even iterate over the items coming out of the stream. Therefore, we include a few minimal convenience methods that are not dependent on any unstable features. Most notably, next + ```rust /// A future that advances the stream and returns the next value. /// @@ -576,6 +586,8 @@ yield could return references to local variables. Given a "detached" or "owned" stream, the generator yield could return things that you own or things that were borrowed from your caller. +### In Iterators + ```rust gen async fn foo() -> Value { yield value; @@ -588,6 +600,8 @@ After desugaring, this would result in a function like: fn foo() -> impl Iterator ``` +### In Async Code + ```rust async gen fn foo() -> Value ``` From 6d06d3fdcc2af581997e8d0e6176c7db54e29b20 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Fri, 10 Jul 2020 10:56:11 -0700 Subject: [PATCH 22/26] Update rfc-drafts/stream.md Co-authored-by: Jake Goulding --- rfc-drafts/stream.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 1ea57325..2840c281 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -348,7 +348,8 @@ We may want a trait similar to this for `Stream`. The `IntoStream` trait would p This trait could look like this: ```rust -pub trait IntoStream where +pub trait IntoStream +where ::Item == Self::Item, { type Item; From 35efa815039d29e6e456249f1b7fe1e74b657f88 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Fri, 10 Jul 2020 07:17:47 -0700 Subject: [PATCH 23/26] removes unnecessary bullet points Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 2840c281..21655a7e 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -39,10 +39,6 @@ about the trait changing during that time ([citation](http://smallcultfollowing. We eventually want dedicated syntax for working with streams, which will require a shared trait. This includes a trait for producing streams and a trait for consuming streams. -## Why is the stream trait defined how it is? -* It is the "pollable iterator" -* [dyn compatibility](https://doc.rust-lang.org/std/keyword.dyn.html) - # Guide-level explanation [guide-level-explanation]: #guide-level-explanation From 4bcc19430f645edabd6909f414670a07cb708397 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Fri, 10 Jul 2020 07:21:19 -0700 Subject: [PATCH 24/26] fixes name of crate Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 21655a7e..613093f0 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -128,7 +128,7 @@ where ## Next method/struct -We should also implement a next method, similar to [the implementation in the futures crate](https://docs.rs/futures-util/0.3.5/src/futures_util/stream/stream/next.rs.html#10-12). +We should also implement a next method, similar to [the implementation in the futures-util crate](https://docs.rs/futures-util/0.3.5/src/futures_util/stream/stream/next.rs.html#10-12). In general, we have purposefully kept the core trait definition minimal. There are a number of useful extension methods that are available, for example, in the futures-stream crate, but we have not included them because they involve closure arguments, and we have not yet finalized the design of async closures. From 642935adc1e9a1cfe04109ec7c85623714678319 Mon Sep 17 00:00:00 2001 From: Nell Shamrell-Harrington Date: Tue, 14 Jul 2020 11:21:35 -0700 Subject: [PATCH 25/26] Update rfc-drafts/stream.md Co-authored-by: Dominik Stolz --- rfc-drafts/stream.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index 613093f0..f358e409 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -586,7 +586,7 @@ that you own or things that were borrowed from your caller. ### In Iterators ```rust -gen async fn foo() -> Value { +gen fn foo() -> Value { yield value; } ``` From b9c8bff570db221abc57480948bb7de5451d7509 Mon Sep 17 00:00:00 2001 From: Nell Shamrell Date: Thu, 16 Jul 2020 11:18:01 -0700 Subject: [PATCH 26/26] adds more info based on feedback Signed-off-by: Nell Shamrell --- rfc-drafts/stream.md | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-) diff --git a/rfc-drafts/stream.md b/rfc-drafts/stream.md index f358e409..e01d60ff 100644 --- a/rfc-drafts/stream.md +++ b/rfc-drafts/stream.md @@ -75,6 +75,8 @@ pub trait Stream { Self: Unpin; } ``` +* For information on why `Self: Unpin` is included in the `next` +method, please see [this discussion on the draft RFC](https://github.com/rust-lang/wg-async-foundations/pull/15#discussion_r452482084). The arguments to `poll_next` match that of the [`Future::poll`] method: @@ -130,9 +132,14 @@ where We should also implement a next method, similar to [the implementation in the futures-util crate](https://docs.rs/futures-util/0.3.5/src/futures_util/stream/stream/next.rs.html#10-12). -In general, we have purposefully kept the core trait definition minimal. There are a number of useful extension methods that are available, for example, in the futures-stream crate, but we have not included them because they involve closure arguments, and we have not yet finalized the design of async closures. +In general, we have purposefully kept the core trait definition minimal. +There are a number of useful extension methods that are available, for example, +in the futures-stream crate, but we have not included them because they involve +closure arguments, and we have not yet finalized the design of async closures. -However, the core methods alone are extremely unergonomic. You can't even iterate over the items coming out of the stream. Therefore, we include a few minimal convenience methods that are not dependent on any unstable features. Most notably, next +However, the core methods alone are extremely unergonomic. You can't even iterate +over the items coming out of the stream. Therefore, we include a few minimal +convenience methods that are not dependent on any unstable features. Most notably, next ```rust /// A future that advances the stream and returns the next value. @@ -225,7 +232,8 @@ Why should we *not* do this? ## Where should stream live? -As mentioned above, `core::stream` is analogous to `core::future`. But, do we want to find some other naming scheme that can scale up to other future additions, such as io traits or channels? +As mentioned above, `core::stream` is analogous to `core::future`. But, do we want to find +some other naming scheme that can scale up to other future additions, such as io traits or channels? # Prior art [prior-art]: #prior-art @@ -439,6 +447,12 @@ pub trait Stream { } ``` +When drafting this RFC, there was [discussion](https://github.com/rust-lang/wg-async-foundations/pull/15#discussion_r451182595) +about whether to implement from_stream for all T where `T: FromIterator` as well. +`FromStream` is perhaps more general than `FromIterator` because the await point is allowed to suspend execution of the +current function, but doesn't have too. Therefore, many (if not all) existing impls of `FromIterator` would work +for `FromStream` as well. While this would be a good point for a future discussion, it is not in the scope of this RFC. + ## Other Traits Eventually, we may also want to add some (if not all) of the roster of traits we found useful for `Iterator`. @@ -480,11 +494,16 @@ There has been much discussion around lending streams (also referred to as attac [Source](https://smallcultfollowing.com/babysteps/blog/2019/12/10/async-interview-2-cramertj-part-2/#the-need-for-streaming-streams-and-iterators) -In a **lending** stream (also known as an "attached" stream), the `Item` that gets returned by `Stream` may be borrowed from `self`. It can only be used as long as the `self` reference remains live. +In a **lending** stream (also known as an "attached" stream), the `Item` that gets +returned by `Stream` may be borrowed from `self`. It can only be used as long as +the `self` reference remains live. -In a **non-lending** stream (also known as a "detached" stream), the `Item` that gets returned by `Stream` is "detached" from self. This means it can be stored and moved about independently from `self`. +In a **non-lending** stream (also known as a "detached" stream), the `Item` that +gets returned by `Stream` is "detached" from self. This means it can be stored +and moved about independently from `self`. -This RFC does not cover the addition of lending streams (streams as implemented through this RFC are all non-lending streams). +This RFC does not cover the addition of lending streams (streams as implemented through +this RFC are all non-lending streams). We can add the `Stream` trait to the standard library now and delay adding in this distinction between the two types of streams - lending and @@ -547,7 +566,8 @@ become non-lending ones? **Coherence** -The impl above has a problem. As the Rust language stands today, we cannot cleanly convert impl Stream to impl LendingStream due to a coherence conflict. +The impl above has a problem. As the Rust language stands today, we cannot cleanly convert +impl Stream to impl LendingStream due to a coherence conflict. If you have other impls like: @@ -569,7 +589,8 @@ Resolving this would require either an explicit “wrapper” step or else some It should be noted that the same applies to Iterator, it is not unique to Stream. -These use cases for lending/non-lending streams need more thought, which is part of the reason it is out of the scope of this particular RFC. +These use cases for lending/non-lending streams need more thought, which is part of the reason it +is out of the scope of this particular RFC. ## Generator syntax [generator syntax]: #generator-syntax