Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prefetch related data in incidents endpoint #837

Merged
merged 2 commits into from
Jul 24, 2024

Conversation

lunkwill42
Copy link
Member

The incidents endpoint does not appear to prefetch tag data when querying incidents. This may cause the response to be magnitudes slower that it needs to be.

For context, fetching 2.5K incidents is 99% faster if tag data is prefetched:

>>> timeit.timeit("""
... for i in x:
...     i.deprecated_tags
... """, setup="x = Incident.objects.all()[:2500]", number=2, globals=locals())
69.45030555600533
>>> timeit.timeit("""
... for i in x:
...    i.deprecated_tags
... """, setup="x = Incident.objects.prefetch_default_related()[:2500]", number=2, globals=locals())
0.4965064429998165
>>>

The difference may not be as stark in the API endpoint itself, as most of the time there is spent serializing the data to JSON (DRF's ModelSerializer is reaally slow). Simply testing the API before and after applying this change seems to indicate only a 36% speedup (which is still significant).

Unless we prefetch related data (i.e. tags), the incidents endpoint may
get ridiculously slow when fetching many incidents.
@lunkwill42 lunkwill42 added the optimization Something is too slow/takes too much memory label Jun 27, 2024
@lunkwill42 lunkwill42 self-assigned this Jun 27, 2024
@lunkwill42 lunkwill42 requested a review from a team June 27, 2024 13:05
Copy link

@codecov-commenter
Copy link

codecov-commenter commented Jun 27, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.81%. Comparing base (cdc23e5) to head (4884639).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #837   +/-   ##
=======================================
  Coverage   84.81%   84.81%           
=======================================
  Files          76       76           
  Lines        3846     3846           
=======================================
  Hits         3262     3262           
  Misses        584      584           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link

Test results

    7 files    616 suites   23m 2s ⏱️
  489 tests   488 ✅ 1 💤 0 ❌
3 423 runs  3 416 ✅ 7 💤 0 ❌

Results for commit 4884639.

@hmpf hmpf merged commit 2e90961 into master Jul 24, 2024
13 checks passed
@hmpf hmpf deleted the feature/incidents-api-prefetch branch July 24, 2024 13:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
optimization Something is too slow/takes too much memory
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants