Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question/Beginner] Simple Diff #248

Closed
bearrito opened this issue Oct 14, 2023 · 5 comments
Closed

[Question/Beginner] Simple Diff #248

bearrito opened this issue Oct 14, 2023 · 5 comments

Comments

@bearrito
Copy link
Contributor

Hello,

I just started looking at this project. Looks great on first glance.

I think I cannot see the forest for the trees, with some of these examples.

Assuming I have two .raw files Old.raw and New.raw. How could I do a simple test (all local) to show what files would have to be downloaded for New.raw

@folbricht
Copy link
Owner

You would first chunk both files into a chunk store, creating 2 indexes. This typically happens on the "server", where you host the chunk-store and have access to the image files.

desync make -s store old.raw.caibx old.raw
desync make -s store new.raw.caibx new.raw

You can then use the info command to tell you what would have to be retrieved from the store that doesn't already exist in the seed (old.raw.caibx+old.raw). This can be run anywhere that has access to the old file, the old index, and a link to the store.

desync info --format=plain -s store --seed=old.raw.caibx new.raw.caibx

Blob size: 2726297600
Size of deduplicated chunks not in seed: 1965580
Size of deduplicated chunks not in seed nor cache: 1965580
Total chunks: 26497
Unique chunks: 21052
Chunks in store: 21052
Chunks in seed: 21030
Chunks in cache: 0
Chunks not in seed nor cache: 22
Chunk size min: 16384
Chunk size avg: 65536
Chunk size max: 262144

In the above example, 22 chunks would have to be downloaded from the store to build the new.raw file, those would add up to 1.9MB.

Does this help?

@bearrito
Copy link
Contributor Author

This was a big help. Closing as helpful.

@bearrito
Copy link
Contributor Author

@folbricht Is the size listed compressed or uncompressed? I thought it was uncompressed, but tried looking at the code, but couldn't tell.

@RyuzakiKK
Copy link
Contributor

The size reported by desync info is the size in Bytes of the uncompressed chunks.

@bearrito
Copy link
Contributor Author

@RyuzakiKK Thank you for confirming.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants