Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decrease allocations in encode loop. #86

Merged
merged 1 commit into from
Apr 5, 2019

Conversation

Lukasa
Copy link
Contributor

@Lukasa Lukasa commented Apr 5, 2019

Motivation

While investigating performance of HTTP/2 I noticed a substantial number
of allocations occuring in a hot loop in header compression. Specifically
I set up a simple harness that encoded headers 100 times, and then ran
that harness 10k times. This harness ended up performing 17 million
allocations, which is frankly a bit too high!

The fact that the number of allocations was in the millions suggested that
we were allocating some resources per header encode. That's not good:
assuming the target ByteBuffer is big enough we should not need to
perform new allocations to encode headers.

The allocations came from two places. Firstly, we were allocating in the
HeaderTableStorage.indices(matching:) function. This was because that
operation was returning a lazy collection, which required multiple heap
allocations for the closure contexts. Those needed to be eliminated.

Secondly, we were triggering a CoW of the ByteBuffer passed in to
encode(). This is the result of a complex bit of control flow that
fundamentally meant that the buffer was held in two places in that
method.

Removing these two sources of allocations dropped the count in my test
from 17 million to 176k, a decrease of 100x. We also saw a speed boost
of nearly 66% in terms of runtime of the microbenchmark.

Modifications

  • Removed HeaderTableStorage.indices(matching:)
  • Implemented `HeaderTableStorage.closestMatch(name:value:), containing
    the repeated logic of the body of the two loops over the header
    indices.
  • Used swap() and an empty byte buffer to ensure that encode does not
    cause a CoW operation.

Result

Much faster and more memory efficient header encoding.

While investigating performance of HTTP/2 I noticed a substantial number
of allocations occuring in a hot loop in header compression. Specifically
I set up a simple harness that encoded headers 100 times, and then ran
that harness 10k times. This harness ended up performing 17 million
allocations, which is frankly a bit too high!

The fact that the number of allocations was in the millions suggested that
we were allocating some resources per header encode. That's not good:
assuming the target `ByteBuffer` is big enough we should not need to
perform new allocations to encode headers.

The allocations came from two places. Firstly, we were allocating in the
`HeaderTableStorage.indices(matching:)` function. This was because that
operation was returning a lazy collection, which required multiple heap
allocations for the closure contexts. Those needed to be eliminated.

Secondly, we were triggering a CoW of the `ByteBuffer` passed in to
`encode()`. This is the result of a complex bit of control flow that
fundamentally meant that the buffer was held in two places in that
method.

Removing these two sources of allocations dropped the count in my test
from 17 million to 176k, a decrease of 100x. We also saw a speed boost
of nearly 66% in terms of runtime of the microbenchmark.

- Removed `HeaderTableStorage.indices(matching:)`
- Implemented `HeaderTableStorage.closestMatch(name:value:), containing
   the repeated logic of the body of the two loops over the header
   indices.
- Used swap() and an empty byte buffer to ensure that encode does not
   cause a CoW operation.

Much faster and more memory efficient header encoding.
@Lukasa Lukasa added the 🔨 semver/patch No public API change. label Apr 5, 2019
@Lukasa Lukasa added this to the 1.0.2 milestone Apr 5, 2019
@Lukasa Lukasa requested a review from weissi April 5, 2019 14:57
Copy link
Member

@weissi weissi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great work @Lukasa

@weissi weissi merged commit 70ac404 into apple:master Apr 5, 2019
@Lukasa Lukasa deleted the cb-remove-allocations branch April 5, 2019 15:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🔨 semver/patch No public API change.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants