Improving tokio_stream with GAT support

[jmjoy]

Currently, tokio_stream exports the Stream trait from futures_core::stream. This trait was designed before Generic Associated Types (GATs) were available in Rust. As a result, there are some limitations in certain scenarios where GAT capabilities would provide more flexibility and expressiveness.

Current Limitations

The pre-GAT design of Stream means that the trait cannot easily handle streaming references with different lifetimes or other generic use cases that GATs are designed to solve. This is particularly limiting when trying to create streams that yield references to internal data with different lifetimes.

Current Implementation

Here's what the current implementation looks like:

pub trait Stream {
    type Item;

    // Required method
    fn poll_next(
        self: Pin<&mut Self>,
        cx: &mut Context<'_>,
    ) -> Poll<Option<Self::Item>>;

    // Provided method
    fn size_hint(&self) -> (usize, Option<usize>);
}

GAT-Based Improved Version

Here's how the trait could be redesigned using GATs:

pub trait Stream {
    type Item<'a> where Self: 'a;

    // Required method
    fn poll_next(
        self: Pin<&mut Self>,
        cx: &mut Context<'_>,
    ) -> Poll<Option<Self::Item<'_>>>;

    // Provided method
    fn size_hint(&self) -> (usize, Option<usize>);
}

Benefits

The GAT-based approach offers several advantages:

1. Flexible lifetime handling: Streams can yield references with lifetimes that aren't tied to the stream's own lifetime.
2. More expressive APIs: Can model more complex relationships between the stream and its yielded items.
3. Better ergonomics: No need for workarounds like owning wrapper types when dealing with references.

[Darksonn]

The tokio-stream crate was intended as a temporary thing until the standard library adds the Stream trait. If you want to improve the trait, I suggest you get involved with the efforts in the stdlib.