-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Compression Support #282
Comments
@ericmcbride yeah, compression is something we should 100% add. I am curious what type of compression you are using with grpc-go? Is it at the connection level or is it per message? I'd love to support this though I am not 100% sure on how to do this yet, so happy to chat about it! Feel free to ping me on discord as well and we can chat if this is something you are interested in contributing :) |
Yeah we just use gzip and enable it as an option. I haven't really went deep into the rabbit hole. At first glance
Makes me thing its per message but this is a quick assumption. Ill dive deeper into the code this weekend and look and id be more then happy to jump on! |
I think if compression is in connection-level, need to check peer support compression or not, but it makes developers can focus on the real job, instead of 'how to set the framework correctly'. And if we could choose which rpc call enable compression, which one disable, it's very good, because when we just send some little bytes, compress them doesn't reduce time, sometimes even make data bigger. However, if we need to send a lot of data, like sending a photo, compress it may reduce a lot of transport time |
I assume we will want to change our generated clients to be more like a builder api that can allow users to set compression per service. |
In my tonic-based client-server app there is a need in compression of grpc stream data. |
Here's a link to the gRPC c++ compression documentation They outline test cases that exemplify the behaviour of compression
|
tower-http has compression and decompression middlewares that support any |
I think adding support for compression is something I would like to look into next. It seems we already have #476 which gets us at least part of the way there. However there is one thing I don't quite understand. In requirement 6 listed above
Why is there suddenly a Why are the My initial plan was to make a tower middleware that compresses (or decompresses) bodies one chunk/frame at a time (what you get from |
@davidpdrsn I'm far from an expert but here is how I interpreted the specification: Essentially the headers are giving the information on "how would it be compressed if it was compressed" and then each message (there can be multiple on a stream) can be compressed or not. Each message having the structure I'm not sure how it can be done at the http tower level as it would need to interpret or produce grpc frames and the handling is very grpc protocol specific.
This section for me is to be interpreted that if we encounter the compressed flag but can't know the encoding (no header provided or just identity) we should fail with internal error (instead of assuming gzip or accepting identity + compressed flag) |
Ah yes. That makes sense! Thanks 😊 |
Feature Request
Add compression to Tonic
Crates
https://github.com/BurntSushi/rust-snappy
Motivation
We use gzip (default) in our Go GRPC services. Would like to normalize this across all services. There are other compression options such as snappy. I noticed burntsushi has a nice snappy implementation in rust.
Proposal
Add some level of compression for larger payloads.
Id be more then happy to help with the implementation of this. Been wanting to get my hands dirty on some more open source projects. I don't know if any ideas have been floated or not about this, or if it belongs here or in tower.
The text was updated successfully, but these errors were encountered: