-
Notifications
You must be signed in to change notification settings - Fork 160
refactor: caching in jsonld processor #2487
refactor: caching in jsonld processor #2487
Conversation
Signed-off-by: Andriy Holovko <[email protected]>
Codecov Report
@@ Coverage Diff @@
## main #2487 +/- ##
==========================================
- Coverage 89.29% 89.28% -0.02%
==========================================
Files 250 250
Lines 18851 18822 -29
==========================================
- Hits 16833 16805 -28
+ Misses 1187 1186 -1
Partials 831 831
Continue to review full report at Codecov.
|
Does this fix #1833? |
It looks like the mentioned issue describes two problems:
targeted by this PR
I am trying to reproduce this panic with the old code. And then check it with the updated one (but I think that the problem has gone). @llorllale plz correct me if I didn't get it right. |
During the discussion with @kdimak, we decided that the caching functionality requires more consideration and also should account for other issues like the one mentioned by George. I am closing the current PR as it doesn't bring enough benefits to justify the risk of breaking the core stuff. |
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code in the framework. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code in the framework. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code in the framework. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code in the framework. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code in the framework. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This change creates a local JSON-LD Document Cache loader using a cache of type *sync.Map to avoid 'error on cache writes' panics. This change also introduces the 'benchmark' Make target to support benchmarking code in the framework. closes hyperledger-archives#1833 for the fatal error mentioned in the comment Also related and closes the discussion about the same panic topic in hyperledger-archives#2487 Signed-off-by: Baha Shaaban <[email protected]>
This PR is a proposal on refactoring caching functionality in jsold processor.
There are 2 major issues that I have with the current approach:
Using
WithDocumentLoaderCache
might look like a robust solution but in practice, it requires a lot of boilerplate code (examples in Trustbloc projects: here, here, and similar should be added to edge-agent).Cache is preloaded in one case and is not in another: here and here. It's also not necessarily clear that under the hood framework adds some additional opts to ones that you have specified explicitly. There is also hardcoded usage of
ld.NewCachingDocumentLoader
.The latter is a problem because I might want to useUPD: existing code has guards against such scenario hereld.NewCachingDocumentLoader
with preloaded contexts and be able to pass it to the processor usingWithDocumentLoader(...)
- the current implementation will wrap it with one moreld.NewCachingDocumentLoader
.The current PR simplifies logic and adds more consistency to API.
ld.NewCachingDocumentLoader
with predefined contexts in the cache is used as a default. You can override it to any loader you want with options. I also movedcache.go
fromproof
package tojsonld
- for me it looks more like a processor concern.Signed-off-by: Andriy Holovko [email protected]