This repository has been archived by the owner on Nov 23, 2024. It is now read-only.
Add a performance spec #69
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Strip some whitespace and log the class since we're already logging the message. Before `GlobalUID error: execution expired Timed out establishing a connection to test_id_server_1` After `GlobalUID error: GlobalUid::ConnectionTimeoutException Timed out establishing a connection to test_id_server_1`
mriddle
force-pushed
the
mriddle-refactor
branch
from
a89a704 to
999cf63
Compare
bquorning
reviewed
Apr 2, 2020
There was a problem hiding this comment.
Good stuff!
I was thinking, since the performance test is quite a bit slower than, and outputs a lot more than the . from all other test, if we should run it as a separate rake task? Rakefile could say e.g.
task :default => ['test', 'performance_test']
What do you think?
The defaults being set differed to those set by the test_helper.
Not being used yet but it will in a follow-up PR
To opt out of global ID you must set `use_global_uid` to `false` in the migration and `disable_global_uid` on the model. Without this, the `before_create` hook in the `ActiveRecordExtension` will still be executed
mriddle
force-pushed
the
mriddle-refactor
branch
5 times, most recently
from
35ddc6a to
c7c0f94
Compare
mriddle
force-pushed
the
mriddle-refactor
branch
from
c7c0f94 to
81f28d6
Compare
bquorning
approved these changes
Apr 6, 2020
This spec highlights the overhead created by using GlobalUID. The
results will vary per run but it will give you an idea and highlight
performance issues if they're introduced.
Output from my local machine:
```
Warming up --------------------------------------
WithGlobalUID 77.000 i/100ms
WithoutGlobalUID 127.000 i/100ms
Calculating -------------------------------------
WithGlobalUID 923.498 (± 8.3%) i/s - 4.620k in 5.041687s
WithoutGlobalUID 1.330k (± 6.5%) i/s - 6.731k in 5.081601s
Comparison:
WithoutGlobalUID: 1330.4 i/s
WithGlobalUID: 923.5 i/s - 1.44x slower
```
As you can see, the performance overhead is noticable but minimal. With
GlobalUID we're able to generate around 923 records per second. This is
not indicative of the performance you'll get in production, where the
databases are more powerful and there's network latency.
The assertion has been commented out due to a lot of variance between
runs, things I've tried:
* Increase the warm-up (up to 5 sec) and time intervals (up to 20 sec)
* Use `kalibera` with `benchmark-ips` and play with confidence levels
* Increase the threshold to 2 seconds, it passes but it's not really
valuable
* Run many times and check an average or check that one has the result
we're after, caters for outliers but also not valuable. Will miss
upward trends
We could look at commiting the results and keeping track in source
control, that would give us visibility when/if performance issues are
introduced but I'm parking that decision for another day.
Nested files under their respective directories now that we've got a couple tests and the support files are used in more than one place
mriddle
force-pushed
the
mriddle-refactor
branch
from
81f28d6 to
95ea2f5
Compare
bquorning
approved these changes
Apr 6, 2020
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There's no mention of the overhead that this gem introduces and I wanted a benchmark before adding the validation code in #63
This PR also includes some minor refactoring, more detail can be found in the associated commits.