Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Review number of fractal-server API calls #9

Closed
tcompa opened this issue Jul 18, 2024 · 6 comments · Fixed by #12
Closed

Review number of fractal-server API calls #9

tcompa opened this issue Jul 18, 2024 · 6 comments · Fixed by #12

Comments

@tcompa
Copy link
Collaborator

tcompa commented Jul 18, 2024

If while browsing a zarr multiscale we need to open N chunk files, how many calls are made to the current-user endpoint?
We should make sure that this remains under control.

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 18, 2024

TO DO: let's rechunk the original testing zarr, so that it has a lot of small chunks.

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 18, 2024

TO DO: let's rechunk the original testing zarr, so that it has a lot of small chunks.

Ref #10

@zonia3000
Copy link
Collaborator

zonia3000 commented Jul 22, 2024

I've added the cache-manager package to cache the result of current user calls. We are using just it's core (in-memory implementation).

The tests using the rechunked zarr with a lot of chunks showed a visible improvements. It also appears that Node 20 is always faster than Node 16.

Node 20 (vizarr loading 818 requests):

  • with cache: 4.06s
  • without cache: 5.66s

Node 16 (vizarr loading 818 requests):

  • with cache: 6.54s
  • without cache: 7.42s

Cache expiration time can be configured using the environment variable CACHE_EXPIRATION_TIME. The default value is 60 seconds.

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 22, 2024

Great!

Could you also specify what do these times refer to? Is it something like loading the first vizarr view?

@zonia3000
Copy link
Collaborator

Yes, I'm loading this page, using the rechunked test zarr:

image

@tcompa
Copy link
Collaborator Author

tcompa commented Jul 23, 2024

I tested this myself as well, as part of #15, and I confirm it works as expected.
I loaded the many-chunks version of our testing data, and kept browsing it for a while. And fractal-server logs show that the API was only called once per minute

2024-07-23 13:28:23,783 - uvicorn.access - INFO - 127.0.0.1:33950 - "GET /auth/current-user/ HTTP/1.1" 200
2024-07-23 13:29:26,782 - uvicorn.access - INFO - 127.0.0.1:40774 - "GET /auth/current-user/ HTTP/1.1" 200
2024-07-23 13:30:26,830 - uvicorn.access - INFO - 127.0.0.1:50488 - "GET /auth/current-user/ HTTP/1.1" 200

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants