Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Api::log_stream to return AsyncBufRead #1235

Merged
merged 1 commit into from
Jul 5, 2023

Conversation

aryan9600
Copy link
Contributor

@aryan9600 aryan9600 commented Jul 2, 2023

Motivation

The current log_stream implementation returns a stream of bytes that aren't newline buffered. This can cause a visual mess, when for example, there are multiple log lines in one chunk of the stream.

Solution

Modify kube_client::api::subresource::Api::log_stream() to return an async buf reader (futures::io::AsyncBufRead) that can read logs asynchronously. Users can call call .lines() on it to get a newline buffered stream of logs.

Fixes #1160

Copy link
Member

@clux clux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this looks sensible to me at a quick look. thanks for tackling this.
some quick comments.

@clux clux added the changelog-change changelog change category for prs label Jul 2, 2023
@aryan9600 aryan9600 force-pushed the log-stream branch 2 times, most recently from 0281b8f to 32ae94f Compare July 2, 2023 15:24
@aryan9600 aryan9600 requested a review from clux July 2, 2023 15:26
@codecov
Copy link

codecov bot commented Jul 2, 2023

Codecov Report

Merging #1235 (ae6b9ee) into main (bd14267) will increase coverage by 0.00%.
The diff coverage is 90.00%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1235   +/-   ##
=======================================
  Coverage   71.93%   71.93%           
=======================================
  Files          71       71           
  Lines        5725     5727    +2     
=======================================
+ Hits         4118     4120    +2     
  Misses       1607     1607           
Impacted Files Coverage Δ
kube-client/src/client/mod.rs 70.52% <75.00%> (-0.25%) ⬇️
kube-client/src/api/subresource.rs 51.96% <100.00%> (ø)
kube-client/src/lib.rs 93.78% <100.00%> (ø)

... and 1 file with indirect coverage changes

@nightkr
Copy link
Member

nightkr commented Jul 2, 2023

I think this is one of those cases that makes sense and we should make it easy.. but we shouldn't make it required.

If you're writing a tail for a single container then you'd probably want to stream in all data as soon as it becomes available, whereas if you're trying to merge the output of multiple then line-buffering would probably be less confusing.

It would probably be possible to have a .lines() combinator of some kind on the impl Stream<Item = Result<Bytes>> returned by request_text_stream()

@clux

This comment was marked as outdated.

@Dav1dde
Copy link
Member

Dav1dde commented Jul 2, 2023

Is it not possible to expose a AsyncRead or better AsyncBufRead, instead of a stream, which would give us .lines() for free?

@nightkr
Copy link
Member

nightkr commented Jul 2, 2023

I think that would be ideal, yes.

@aryan9600 aryan9600 changed the title modify request_text_stream to return a stream of log lines modify log_stream to return an async buf reader Jul 3, 2023
@aryan9600
Copy link
Contributor Author

changed the implementation to return an AsyncBufRead 👍

Copy link
Member

@Dav1dde Dav1dde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if I think these are good changes, it's now a breaking change. @clux

@aryan9600
Copy link
Contributor Author

don't know why clippy is complaining, these changes don't touch any of the reported code blocks.

@Dav1dde
Copy link
Member

Dav1dde commented Jul 3, 2023

don't know why clippy is complaining, these changes don't touch any of the reported code blocks.

Unlucky timing, nightly must have added more lints, see #1238

I'll try to get them fixed separately, you can just ignore them here

@aryan9600 aryan9600 requested a review from Dav1dde July 3, 2023 19:30
@aryan9600 aryan9600 force-pushed the log-stream branch 2 times, most recently from 57ede7a to a84f9f8 Compare July 3, 2023 22:44
Copy link
Member

@clux clux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM now. Have left a minor doc suggestion, and leaving open for David.

Copy link
Member

@Dav1dde Dav1dde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the changes, I had some ideas for the docs, let me know what you think.

Also I think we should rename request_text_stream since we're already breaking the API.

Modify `kube_client::api::subresource::Api::log_stream()` to return an
async buf reader (`futures::io::AsyncBufRead`) that can read logs
asynchronously. Users can call call `.lines()` on it to get a newline
buffered stream of logs using `futures::io::AsyncBufReadExt`.

Signed-off-by: Sanskar Jaiswal <[email protected]>
@clux clux changed the title modify log_stream to return an async buf reader Change Api::log_stream to return AsyncBufRead Jul 5, 2023
@clux clux merged commit 9fdd27c into kube-rs:main Jul 5, 2023
@clux clux added this to the 0.84.0 milestone Jul 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog-change changelog change category for prs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add option to emit stream of logs where each chunk represent a log line
5 participants