Add DisableCompression in http.Transport to override auto-decompress. #752
Conversation
harshavardhana
force-pushed
the
accept-encoding
branch
4 times, most recently
from
a19bdd8 to
eabacd4
Compare
| req.Header.Set("Content-Encoding", strings.Join([]string{contentEncoding, streamingEncoding}, ",")) | ||
| } else { | ||
| req.Header.Set("Content-Encoding", streamingEncoding) | ||
| } |
There was a problem hiding this comment.
instead you can just do:
req.Header.Add("Content-Encoding", streamingEncoding)
There was a problem hiding this comment.
Oops yes forgot :P @krishnasrinivas
harshavardhana
force-pushed
the
accept-encoding
branch
from
eabacd4 to
3fdfd5a
Compare
api.go
Outdated
| // uncompress the gzip stream if we were the layer that | ||
| // requested it. "" | ||
| // | ||
| req.Header.Set("Accept-Encoding", "gzip") |
There was a problem hiding this comment.
Aren't we here literally saying: hey server, we can accept content in compressed format to gain bandwidth, now if the server supports that feature, will the client really be able to decompress content before delivering it to our users ?
Besides.. I am noticing this weird behavior, an empty Content-Encoding: header returned by AWS S3, but it seems it has nothing to do with this PR.
GET /presignedgetobject.go HTTP/1.1
Host: vadmeste.s3.amazonaws.com
User-Agent: Minio (linux; amd64) minio-go/3.0.0
Accept-Encoding: gzip
Authorization: AWS4-HMAC-SHA256 Credential=**REDACTED**/20170712/us-east-1/s3/aws4_request, SignedHeaders=accept-encoding;host;if-match;x-amz-content-sha256;x-amz-date, Signatu
re=**REDACTED**
If-Match: "6795a04d7e56a596c4cacafc0c148ae8"
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
X-Amz-Date: 20170712T151340Z
HTTP/1.1 200 OK
Content-Length: 1736
Accept-Ranges: bytes
Content-Encoding:
Content-Type: application/octet-stream
Date: Wed, 12 Jul 2017 15:13:41 GMT
Etag: "6795a04d7e56a596c4cacafc0c148ae8"
Last-Modified: Wed, 12 Jul 2017 15:12:55 GMT
Server: AmazonS3
X-Amz-Id-2: jt7yeuBacaGCBGdU1cJ1BVGZTAKZdWy5u3u7expyF61LyTBiARPqC/sPpSOQbHbC6+3HdTAoafw=
X-Amz-Request-Id: 9071463091587F0D
Yes that is correct indication but we are doing that to avoid auto-decompress for copying and streaming. Such that out GetObject() gets a compressed stream and the caller can decide on decompressing it or un-compressing it. minio-go client will not do any data manipulation. |
Found a way to avoid setting any additional headers instead just use |
harshavardhana
force-pushed
the
accept-encoding
branch
from
3fdfd5a to
0e95d9a
Compare
harshavardhana
changed the title
api.go
Outdated
| // | ||
| // Refer: | ||
| // https://golang.org/src/net/http/transport.go?h=roundTrip#L1843 | ||
| DisableCompression: true, |
There was a problem hiding this comment.
We also need to do this in mc since mc sets its custom transport (primarily to satisfy --insecure flag). But I am OK with this.
There was a problem hiding this comment.
Yes we need to. @vadmeste
harshavardhana
force-pushed
the
accept-encoding
branch
3 times, most recently
from
8ec0c4f to
050e76c
Compare
harshavardhana
force-pushed
the
accept-encoding
branch
from
050e76c to
828890e
Compare
harshavardhana
force-pushed
the
accept-encoding
branch
from
828890e to
0689ec0
Compare
|
ping @vadmeste waiting on your review. |
Fixes minio/mc#2209