Stop writing duplicated blobs to storage for routerlicious driver. #592
Conversation
jatgarg
changed the title
packages/drivers/routerlicious-socket-storage/src/documentStorageService.ts
Outdated
Show resolved
Hide resolved
|
ODSP part is the most interesting and missing. |
Yes, first wanted to get this done. So that I get more idea. |
|
|
||
| /** | ||
| * Document access to underlying storage for routerlicious driver. | ||
| */ | ||
| export class DocumentStorageService implements IDocumentStorageService { | ||
|
|
||
| private readonly blobsPathCache = new Map<string, string>(); |
There was a problem hiding this comment.
The cache currently may include loose git objects for nacked summaries. In normal summarize case, there is a tiny window of opportunity for them to be garbage collected before the server commits them and moves the pointer to that commit.
Here, we would be increasing that window up to the life of the summarizer container. Probably fine for now, but we might like to invalidate the cache for nacks or change it so we enable on ack.
There was a problem hiding this comment.
I've opened Issue # 598 to follow up here.
Note - it might be there is no work to do here, if R11S promises to never GC blobs.
In reply to: 346028389 [](ancestors = 346028389)
packages/drivers/routerlicious-socket-storage/src/documentStorageService.ts
Show resolved
Hide resolved
packages/drivers/routerlicious-socket-storage/src/documentStorageService.ts
Show resolved
Hide resolved
|
The cache should be initialized during load by populating it in the getSnapshotTree call. It can go through the resulting tree and add all blob ids to the cache. |
|
I will follow up on initializing the cache. |
|
Linking to issue #398 |
* Stop writing duplicated blobs to storage for routerlicious driver. (#592) * stop writing duplicated blobs to storage * change hashing logic * add assert * populate cache with the summary blobs (#626) * populate cache with the summary blobs * add in cache while reading tree * rename * blob deduping for odsp driver (#639) * blob deduping for odsp driver * pr suggestions * update map in get latest * have 2 caches with latest and prev caching * populate cache in blob read * change comment * make map local * make local * local
) * Stop writing duplicated blobs to storage for routerlicious driver. (#592) * stop writing duplicated blobs to storage * change hashing logic * add assert * populate cache with the summary blobs (#626) * populate cache with the summary blobs * add in cache while reading tree * rename * blob deduping for odsp driver (#639) * blob deduping for odsp driver * pr suggestions * update map in get latest * have 2 caches with latest and prev caching * populate cache in blob read * change comment * make map local * make local * local
1.) Stop writing duplicated blobs to storage for routerlicious driver.