Summarizer: Dedup blobs with same content

[vladsud]

Reducing number of blobs we sent to storage improves performance of the system.
Ideally runtime should be capable of doing it across the board, in efficient way, but here are couple concrete scenarios that cover 99% of the current need:

1 : we produce same attributes blob for every map, or every directory, or every shared string. It would be great for all of them to point to same baseId across components.

channelContext.ts

export function snapshotChannel(channel: IChannel, baseId: string | null) {
    const snapshot = channel.snapshot();

    // Add in the object attributes to the returned tree
    const objectAttributes = channel.attributes;
    snapshot.entries.push(new BlobTreeEntry(".attributes", JSON.stringify(objectAttributes)));

    // If baseId exists then the previous snapshot is still valid
    snapshot.id = baseId;

    return snapshot;
}

2 : It would be great if some dedup was happening for blobs within DDS, even if DDS changed and needs to regenerate blobs. For example, imagine that SharedString implements more stable separation of header & body blobs. While it itself does not track (today) what changed (as long as something changed in shared string), if it was splitting content in stable way and nothing changed in body , it would be great for runtime to notice it and reuse prior blob. It's not ideal (as we spend time generating content for nothing), but it would be way better than what we have today, as we would not transfer bits over wire.

@arinwt - FYI

[vladsud]

Tracks re-enabling blob dedupping work by Jatin

[vladsud]

Checked into 0.14