-
Notifications
You must be signed in to change notification settings - Fork 992
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add sticky caching for release descriptions #3745
Conversation
This involves quite a bit of changes: * Two new fields on the Release model, `rendered_description` and `renderer_version`. * A new property (not field) on the Releaes model, `html_description`, which renders and caches the release description. * Removal of the `readme` filter. * New `camoify` filter separated from the former `readme` filter. * New `warehouse.utils.readme` separated from the former `readme` filter. I am not totally convinced this is the best approach - I am more than open to moving things around. Reviewers, feel free to send commits to this branch.
I don't really like the "check for cache, serve or render inline and cache" approach here, because AIUI part of the problem here is that rendering is taking a long time, so while the cache solves it most of the time, it doesn't solve it for that initial page load. Unfortunately, that leaves us in a bit of a sticky position, because of the initial render. For subsequent re-renders we can pretty easily serve the existing rendered value under the re-render has been successful, but the initial render you really only have a few choices:
Of these, I think I prefer (1). It might make the upload route slower, but it's already pretty slow so that's not really much of a regression. It also has the nice benefit of allowing us to reject the upload if the passed in value doesn't render successfully. That's not a huge deal for markdown, since everything is valid Markdown, but it's a bigger deal for people using rst. |
We can't do it until pypi-legacy is shutdown, but we could also take this opportunity to break long_descriptions and their rendered html into a separate model. Release.long_description is a brute and dramatically complicates access to the Release table. |
Does it really? To be clear, I'm not against it, but now that the |
So yeah, @dstufft has most of the same reservations I had about this: it's not ideal, but it's what we could do if we wanted to do this today. I'm totally fine to wait until legacy is shutdown and go with (1). The great news is that a lot of the changes in this PR can be done ahead of time (such as splitting out the rendering logic and camoify). I'll send PRs to do that and close this one for now. Let's revisit in May (maybe during PyCon?) and close this loop. |
We only need to wait for legacy if we want to move |
Oh, right, because uploads don't go through legacy anymore. Fair enough. Still going to split this into a few PRs. |
This involves quite a bit of changes:
rendered_description
andrenderer_version
.html_description
, which renders and caches the release description.readme
filter.camoify
filter separated from the formerreadme
filter.warehouse.utils.readme
separated from the formerreadme
filter.I am not totally convinced this is the best approach - I am more than open to moving things around. Reviewers, feel free to send commits to this branch.
Partially resolves #3739.