Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return length from StatelessDeflate #222

Closed
wants to merge 1 commit into from
Closed

Return length from StatelessDeflate #222

wants to merge 1 commit into from

Conversation

nhooyr
Copy link

@nhooyr nhooyr commented Feb 14, 2020

Means users who use it in a io.Writer over StatelessWriter
for adjusting the dictionary do not need to think about
the length.

Also fixes a bug in gzip.Writer where because
stateless writer wasn't returning n, len(p) was always returned
even when an error occured.

See #216 (comment)

Means users who use it in a io.Writer over StatelessWriter
for adjusting the dictionary do not need to think about
the length.

Also fixes a bug in gzip.Writer where because
stateless writer wasn't returning n, len(p) was always returned
even when an error occured.

See #216 (comment)
@klauspost
Copy link
Owner

I still don't see why you need it and it doesn't keep its promise. I don't want to hunt down bugs because the API promises something that is wrong.

Your assumption that you can control this is flawed.

There may have been valid parts of your input written even if an error is returned. There is likely junk on the output that has been written.

Your PR does essentially this:

n := len(in)
err := StatelessDeflate(out, in, eof, dict)
if err != nil {
   n = 0
}

If you get an error your dictionary is useless anyway, since the client may very well be out of sync. Calling this function can very likely result in multiple writes.

If you need this functionality, keep the compressed output in a buffer. That is the only way you can control this.

@nhooyr
Copy link
Author

nhooyr commented Feb 14, 2020

Like you said, I don't need to track it, but putting this simple logic into StatelessDeflate avoids callers having to decide what to do when returning the length of bytes written. E.g. in your gzip writer when in stateless mode, you were always returning len(p) instead of 0 when an error occurred which isn't how it normally behaves or how other compression writers behave. If change was part of StatelessDeflate originally, this bug would never have occurred.

I'll document you shouldn't rely on it but the stdlib doesn't either and it doesn't seem to have caused any problems. Neither do any of your other compression writers afaik.

See https://github.com/golang/go/blob/3eab754cd061bf90ee7b540546bc0863f3ad1d85/src/compress/flate/deflate.go#L548-L561

@klauspost
Copy link
Owner

avoids callers having to decide what to do when returning the length of bytes written

But it does not provide any useful information not already in an err != nil. I will not be adding a useless parameter.

  1. It is a breaking change. I made an exception for you the last time, but not doing it this time
  2. I will have to support it, and I don't want to do that.

You are welcome to add the documentation that something may have been written to the output if an error is returned, but honestly that seems kind of pointless.

gzip writer when in stateless mode, you were always returning len(p) instead of 0 when an error occurred

Sure, that can be fixed, but most most practical purposes anything returned with a non-nil error is just junk. Whatever the value returned is, it will not represent anything useful, so whether it is len(p) or 0 doesn't make much difference to me.

@nhooyr
Copy link
Author

nhooyr commented Feb 14, 2020

Fair enough.

@nhooyr nhooyr closed this Feb 14, 2020
@nhooyr nhooyr deleted the len branch February 14, 2020 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants