Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Knowing when library has been reloaded #729

Open
rgaudin opened this issue Feb 4, 2025 · 0 comments
Open

Knowing when library has been reloaded #729

rgaudin opened this issue Feb 4, 2025 · 0 comments

Comments

@rgaudin
Copy link
Member

rgaudin commented Feb 4, 2025

This is related to #728 but independent.

Use case we have is that kiwix-serve is behind a varnish cache for library.kiwix.org because it's not able to handle the load.
Our cache sure has a time-based expiration for all its entries but it's not relevant here.

Because we frequently publish new ZIM files, we frequently (at most once per hour ATM) regenerate the library XML file.
When we do, we want to invalidate our cache entries related to the Catalog.

Our problem is that we don't know when kiwix-serve has actually reloaded the library and is ready to serve new data.

If we invalidate cache too soon, then chances are an incoming request happens before the refresh and we'd be storing old data into the cache instead of the new one.

To workaround this, we are now waiting 10s after writing the XML file on disk and purging the cache. Of course, that's arbitrary, ugly and fragile.

How could we be informed that the library has been reloaded?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant