Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix occasional hang with http2 goaway frames #3516

Closed

Conversation

ifedorenko
Copy link

Http2Connection was dropping headers of healthy streams
after GOAWAY frame.

Fixes #3422

Http2Connection was dropping headers of healthy streams
after GOAWAY frame.

Fixes square#3422

Signed-off-by: Igor Fedorenko <[email protected]>
@ifedorenko
Copy link
Author

Not sure why Travis build failed, all tests passed in Travis and mvn clean verify works for me locally too (osx, java 8)

Results :
Tests run: 350, Failures: 0, Errors: 0, Skipped: 6

@JakeWharton
Copy link
Collaborator

JakeWharton commented Aug 10, 2017 via email

@@ -620,7 +620,7 @@ public Http2Connection build() {
// Create a stream.
final Http2Stream newStream = new Http2Stream(streamId, Http2Connection.this,
false, inFinished, headerBlock);
lastGoodStreamId = streamId;
if (streamId > lastGoodStreamId) lastGoodStreamId = streamId;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don’t think this works. The lastGoodStreamId refers to the local last good stream, and this is the remote last good stream

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, scratch that. In this case both refer to remote stream IDs.

@@ -725,6 +725,7 @@ private void applyAndAckSettings(final Settings peerSettings) {
synchronized (Http2Connection.this) {
streamsCopy = streams.values().toArray(new Http2Stream[streams.size()]);
shutdown = true;
Http2Connection.this.lastGoodStreamId = lastGoodStreamId;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this cause the stream ID to become smaller?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the line that’s most problematic. The local field lastGoodStreamId refers to remotely-initiated streams, and goAway is telling us an ID for locally-initiated streams

@@ -978,6 +982,33 @@ private void noRecoveryFromErrorWithRetryDisabled(ErrorCode errorCode) throws Ex
assertEquals(0, server.takeRequest().getSequenceNumber());
}

@Test public void responseHeadersAfterGoaway() throws Exception {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice test!

@swankjesse
Copy link
Collaborator

I’m trying to understand exactly what’s going on here, and I’m having a bit of trouble mostly because I haven’t looked at this code for a long time.

Client sends request 1
Client sends request 3
Server sends response 3
Server sends GOAWAY shutdown, saying last client-initiated stream ID is 3
Server sends response headers 1
Client rejects response headers 1 because the connection is shutdown

I think the fix is to move this line in headers()

        if (shutdown) return;

down into the block that runs only if the stream is unknown:

        if (stream == null) {
          if (shutdown) return;

@swankjesse
Copy link
Collaborator

... Just tested it, and that’s sufficient to get the new test to pass. If you’re cool with that, that’s what I’ll merge.

@ifedorenko
Copy link
Author

Yes, this is the scenario that causes 3.8.1 to hang and I confirmed your fix solves the problem when running against nginx. Thank you for looking into this.

@swankjesse
Copy link
Collaborator

Merged manually. Thanks!

@swankjesse swankjesse closed this Aug 18, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants