Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IQSS/11264 - Add pseudo-exponential backoff to dataset page refresh #11269

Conversation

qqmyers
Copy link
Member

@qqmyers qqmyers commented Feb 17, 2025

What this PR does / why we need it: The dataset page currently refreshes every 10 seconds looking for locks to be removed, which is not very useful for long-running locks, e.g. in review, or ingest, publish, etc. with large/many files. This PR adds a pseudo-exponential backoff - in the front-end JavaScript - in the rate of refresh to reduce the load. The constants chose make the updates slightly faster for the first two refreshes and then quickly backs off to 5 minute refreshes (after ~ 90 seconds, the new algorithm will have had fewer total refreshes and will only be refreshing ~once per minute)

Which issue(s) this PR closes:

Special notes for your reviewer: This is a minor improvement, but hopefully reduces server load for in review datasets in particular, as well as long running Globus transfers, pub with many files, etc.

Suggestions on how to test this: Put a dataset in review and check the refresh rate in the browser console - verify it is slowing down.

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Additional documentation:

@qqmyers qqmyers added Size: 3 A percentage of a sprint. 2.1 hours. GDCC: QDR of interest to QDR labels Feb 17, 2025
@ofahimIQSS ofahimIQSS requested a review from sekmiller February 18, 2025 16:10
Copy link
Contributor

@sekmiller sekmiller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine. We could in the future think about making the max interval settable - or try something with varying intervals based on type of lock

@ofahimIQSS ofahimIQSS self-assigned this Feb 19, 2025
@ofahimIQSS
Copy link
Contributor

PR tests passing. No issues found during regression.

@ofahimIQSS ofahimIQSS merged commit e226d86 into IQSS:develop Feb 19, 2025
2 checks passed
@ofahimIQSS ofahimIQSS removed their assignment Feb 19, 2025
@pdurbin pdurbin added this to the 6.6 milestone Feb 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
GDCC: QDR of interest to QDR Size: 3 A percentage of a sprint. 2.1 hours.
Projects
Status: Done 🧹
Development

Successfully merging this pull request may close these issues.

Javascripts loading in a loop with datasets in “in review” status
4 participants