Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Interim performance optimization for large input arg types #2128

Conversation

adamni21
Copy link

@adamni21 adamni21 commented Jul 17, 2024

Description

This PR aims to improve the performance for large input arg types.
Eg. a filter input with many elements in the and/or field.

Discussed in discord: https://discord.com/channels/489127045289476126/498852330754801666/1261317211604123718

Only serves as an interim solution, while eval* is being removed/the filter plugin gets rewritten to be evaluated at execution time instead of planning time.

Performance impact

Vast improvement for the mentioned use case, >500x for a filter with 20 "or filters" on a filter type with ~250 fields.
Maybe a slight performance decrease, for general usage, due to increased operation plan splitting.

Security impact

Unknown (likely none)

Checklist

  • My code matches the project's code style and yarn lint:fix passes.
  • I've added tests for the new feature, and yarn test passes.
  • I have detailed the new feature in the relevant documentation.
  • I have added this feature to 'Pending' in the RELEASE_NOTES.md file (if one exists).
  • If this is a breaking change I've explained why.

I didn't add tests, but it should already be covered by grafast/dataplan-pg/__tests__/queries/conditions/complex-filter.test.graphql

Edit:
Added an almost equivalent of the above test, which specifies the filters via variables, instead of inlining it into the query.

Copy link

changeset-bot bot commented Jul 17, 2024

🦋 Changeset detected

Latest commit: 4e102b1

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages
Name Type
@dataplan/pg Patch
grafast Patch
postgraphile Patch
graphile-build-pg Patch
graphile-utils Patch
pgl Patch
@localrepo/grafast-bench Patch
@dataplan/json Patch
@grafserv/persisted Patch
grafserv Patch
ruru-components Patch
@localrepo/grafast-website Patch
graphile-build Patch
graphile-export Patch
graphile Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Member

@benjie benjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is awesome! Here's some initial changes from a very quick review to help us align on the right direction. In particular, just like with arrays, you need to handle the value not being set - in which case evalKeys() should return null.

grafast/grafast/src/steps/__inputObject.ts Outdated Show resolved Hide resolved
grafast/grafast/src/steps/__inputObject.ts Show resolved Hide resolved
grafast/grafast/src/steps/__trackedValue.ts Outdated Show resolved Hide resolved
grafast/grafast/src/steps/__trackedValue.ts Outdated Show resolved Hide resolved
grafast/grafast/src/steps/__trackedValue.ts Show resolved Hide resolved
grafast/grafast/src/steps/__trackedValue.ts Outdated Show resolved Hide resolved
Copy link
Member

@benjie benjie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work; thanks for this! I've optimized it a bit, handled more edge cases, and added another test to check handling of partial variables (and undefined variables). I've also fixed some existing bugs I noticed at the same time which somehow have flown under the radar until now.

@benjie benjie merged commit 0cc8c76 into graphile:main Jul 17, 2024
18 checks passed
@adamni21
Copy link
Author

adamni21 commented Jul 17, 2024

Excellent work; thanks for this! I've optimized it a bit, handled more edge cases, and added another test to check handling of partial variables (and undefined variables). I've also fixed some existing bugs I noticed at the same time which somehow have flown under the radar until now.

Lovely.
Happy to help 😀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

2 participants