Use pyramid-beaker to cache acl permissions for a specific service and user name

[davidcaron]

I only tested this with the ServiceAPI service type.

Basically, the permissions are cached for a specific service and user. When there are a lot of concurrent requests, adding a cache with a small expiration time of 5 seconds can make a big difference (2-3 times more requests handled per second).

@fmigneault I don't think so, but look if I didn't miss anything regarding permissions that caching could interfere with...

[codecov-io]

Codecov Report

Merging #218 into master will decrease coverage by 0.71%.
The diff coverage is 65.21%.

@@            Coverage Diff             @@
##           master     #218      +/-   ##
==========================================
- Coverage   71.38%   70.67%   -0.72%     
==========================================
  Files          95       96       +1     
  Lines        6912     6993      +81     
==========================================
+ Hits         4934     4942       +8     
- Misses       1978     2051      +73

Impacted Files	Coverage Δ
magpie/adapter/__init__.py	0% <0%> (ø)	⬆️
magpie/__init__.py	90.62% <100%> (+0.3%)	⬆️
magpie/app.py	93.47% <100%> (+0.29%)	⬆️
magpie/models.py	85.71% <50%> (-0.65%)	⬇️
magpie/services.py	43.96% <73.33%> (-0.08%)	⬇️
magpie/owsrequest.py	44.73% <0%> (-0.34%)	⬇️
magpie/helpers/create_users.py	0% <0%> (ø)
magpie/utils.py	62.21% <0%> (+0.17%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5067a16...96b68fb. Read the comment docs.

[fmigneault]

@davidcaron
I have tried testing with similar operations.
We need to add config.include("pyramid_beaker") in the adapter's __init__ since Magpie is not added completely on Twitcher's side.
Also, we need to add the config in Twitcher's ini to make them work in the gateway.

I would also add a warning note about caching timeout as it can make unexpected behaviours quite easily if set too high.

[davidcaron]

Also, we need to add the config in Twitcher's ini to make them work in the gateway.

Does twitcher need to know about beaker? I wouldn't think so, as it's not a required dependency.

I would also add a warning note about caching timeout as it can make unexpected behaviours quite easily if set too high.

I set it up so that no caching is done by default... where should the warning be? In the configuration file or in the docs?

[fmigneault]

@davidcaron

Does twitcher need to know about beaker? I wouldn't think so, as it's not a required dependency.

What I mean is that the config specified in magpie.ini here would actually need to be inside twitcher.ini because that's the file that will be loaded by twitcher although he imports magpie's adapter.

[fmigneault]

@davidcaron

I set it up so that no caching is done by default... where should the warning be? In the configuration file or in the docs?

I think it must be disabled by default, and a comment directly in the ini file should state quite clearly that this can cause differences in responses if the cache timeout is too high.

For example, deleting route access permission will not be immediately effective if the timeout hasn't been reached. This could be very undesirable.

Actually, we should probably enforce a cache invalidation on any POST/PUT/DELETE request that modifies any kind of permission to avoid this.

[fmigneault]

consider adding cache invalidation on updated permission requests

[davidcaron]

I think it must be disabled by default

Yes, it is disabled by default.

Actually, we should probably enforce a cache invalidation on any POST/PUT/DELETE request that modifies any kind of permission to avoid this.

I'll look into it. I'll also add some more documentation. I know that this caching can lead to confusing outcomes when done incorrectly.

[fmigneault]

Excellent!