Initial Loader and driver changes for Data Virtualization #19314

jatgarg · 2024-01-22T17:33:59Z

Description

1.) Add interface to support passing all the snapshot contents from storage service to loader. This represents the entire snapshot.
`export interface ISnapshot {
snapshotTree: ISnapshotTree;

blobContents: Map<string, ArrayBuffer>;
ops: ISequencedDocumentMessage[];

/**
 * Sequence number of the snapshot
 */
sequenceNumber: number | undefined;

/**
 * Sequence number for the latest op/snapshot for the file in ODSP
 */
latestSequenceNumber: number | undefined;
snapshotInNewFormat: true;

}`

2.) Add some APIS to differentiate between the full and partial snapshot contents.

3.) Changes in ODSP driver to add api to fetch both full and partial snapshot.

4.) Changes in Loader to support with both full and partial snapshot and fetch it from storage and pass it to the runtime layer.

packages/common/container-definitions/src/runtime.ts

packages/common/driver-definitions/src/storage.ts

packages/drivers/odsp-driver/src/odspDocumentStorageServiceBase.ts

packages/drivers/odsp-driver/src/odspPublicUtils.ts

packages/loader/driver-utils/src/storageUtils.ts

packages/loader/driver-utils/src/documentStorageServiceProxy.ts

packages/loader/container-loader/src/containerStorageAdapter.ts

packages/loader/container-loader/src/container.ts

vladsud · 2024-01-23T22:27:00Z

packages/loader/container-loader/src/container.ts

+	): Promise<{ snapshot?: ISnapshot | ISnapshotTree; versionId?: string }> {
+		if (
+			this.mc.config.getBoolean("Fluid.Container.FetchSnapshotInNewFormat") === true &&
+			this.service?.policies?.supportNewSnapshotFormat === true


do we need supportNewSnapshotFormat? It feels like presence of getSnapshot API should be enough for that, no?
Maybe there are some cases where we have API but it's NYI (i.e. throws). If possible, it would be great not to have that (and basically have one truth on whether it's supported or not)

I have added comment on the prop in interface, but I just want to have it and we will enable it in the driver once the driver fully supports the new api as there could be time when we have this api, but internally it is not fully implemented. Also, we can just leave a hook here in case someone wants to disbale it from app side, by passing this policy just in case. We can always remove it later.

packages/loader/container-loader/src/container.ts

packages/common/driver-definitions/api-report/driver-definitions.api.md

msfluid-bot · 2024-01-30T21:48:35Z

⯅ @fluid-example/bundle-size-tests: +4.06 KB

Metric Name	Baseline Size	Compare Size	Size Diff
aqueduct.js	507.73 KB	507.99 KB	⯅ +268 Bytes
connectionState.js	680 Bytes	680 Bytes	■ No change
containerRuntime.js	241.75 KB	241.99 KB	⯅ +246 Bytes
loader.js	168.89 KB	170.89 KB	⯅ +2 KB
map.js	46.61 KB	46.62 KB	⯅ +11 Bytes
matrix.js	148.67 KB	148.68 KB	⯅ +11 Bytes
odspDriver.js	95.32 KB	96.31 KB	⯅ +1008 Bytes
odspPrefetchSnapshot.js	41.58 KB	41.68 KB	⯅ +93 Bytes
sharedString.js	167.85 KB	167.87 KB	⯅ +11 Bytes
sharedTree.js	295.02 KB	295.43 KB	⯅ +421 Bytes
Total Size	1.82 MB	1.82 MB	⯅ +4.06 KB

Baseline commit: 4b78959

Generated by 🚫 dangerJS against 4cd9131

…19314) ## Description 1.) Add interface to support passing all the snapshot contents from storage service to loader. This represents the entire snapshot. `export interface ISnapshot { snapshotTree: ISnapshotTree; blobContents: Map<string, ArrayBuffer>; ops: ISequencedDocumentMessage[]; /** * Sequence number of the snapshot */ sequenceNumber: number | undefined; /** * Sequence number for the latest op/snapshot for the file in ODSP */ latestSequenceNumber: number | undefined; snapshotInNewFormat: true; }` 2.) Add some APIS to differentiate between the full and partial snapshot contents. 3.) Changes in ODSP driver to add api to fetch both full and partial snapshot. 4.) Changes in Loader to support with both full and partial snapshot and fetch it from storage and pass it to the runtime layer. --------- Co-authored-by: Jatin Garg <[email protected]>

Initial Loader and driver changes for Data Virtualization

3d1252b

jatgarg requested a review from vladsud January 22, 2024 17:33

jatgarg self-assigned this Jan 22, 2024

jatgarg requested review from msfluid-bot and a team as code owners January 22, 2024 17:34

jatgarg marked this pull request as draft January 22, 2024 17:34

github-actions bot added area: driver Driver related issues area: loader Loader related issues area: odsp-driver area: tests Tests to add, test infrastructure improvements, etc public api change Changes to a public API base: main PRs targeted against main branch labels Jan 22, 2024