Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forest actor_bundles.car.zst asset has a premature end #3485

Closed
elmattic opened this issue Sep 13, 2023 · 4 comments
Closed

Forest actor_bundles.car.zst asset has a premature end #3485

elmattic opened this issue Sep 13, 2023 · 4 comments

Comments

@elmattic
Copy link
Contributor

elmattic commented Sep 13, 2023

Found out that our current actor_bundles.car.zst file in Forest repository doesn't deflate well:

$ zstd -d actor_bundles.car.zst
ctor_bundles.car.zst : 0 B...     ctor_bundles.car.zst : Read error (39) : premature end 
$ zstd --test actor_bundles.car.zst 
ctor_bundles.car.zst : 0 B...     ctor_bundles.car.zst : Read error (39) : premature end 

We should investigate why this is happening.

Originally posted by @elmattic in #3461 (comment)

@aatifsyed
Copy link
Contributor

Looking at the file history, this has always been broken... https://github.com/ChainSafe/forest/commits/main/assets/actor_bundles.car.zst

for commit in e700f66aba63367f8ec5c86d932dd34ef9c6e247 9673ac4aac101ac016318e0f27592bfd585a24e6; do
    git lfs fetch origin "$commit" >/dev/null
    pointer=$(git cat-file -p "$commit:assets/actor_bundles.car.zst")
    oid=$(echo "$pointer" | rg 'oid sha256:(.*)' --replace '$1')
    blob=$(fd "$oid" .git/lfs)

    set -x
    git show --summary "$commit"
    ls -l "$blob"
    file "$blob"
    zstd --test "$blob"
    set +x
    printf '\n'
done
+ git show --summary e700f66aba63367f8ec5c86d932dd34ef9c6e247
commit e700f66aba63367f8ec5c86d932dd34ef9c6e247
Author: hanabi1224 <[email protected]>
Date:   Mon Aug 14 18:22:22 2023 +0800

    fix: move actor bundles to git lfs (#3360)

 create mode 100644 .gitattributes
 create mode 100644 assets/actor_bundles.car.zst
 create mode 100755 assets/ci_download.sh
+ ls -l .git/lfs/objects/fd/f1/fdf134ea31dffeb7e43b97b2aeaf60f57cebb6293a7ec9024ce74fe83ae84eff
-rw-r--r-- 1 aatif aatif 2438387 Aug 14 11:53 .git/lfs/objects/fd/f1/fdf134ea31dffeb7e43b97b2aeaf60f57cebb6293a7ec9024ce74fe83ae84eff
+ file .git/lfs/objects/fd/f1/fdf134ea31dffeb7e43b97b2aeaf60f57cebb6293a7ec9024ce74fe83ae84eff
+ zstd --test .git/lfs/objects/fd/f1/fdf134ea31dffeb7e43b97b2aeaf60f57cebb6293a7ec9024ce74fe83ae84eff
.git/lfs/objects/fd/f1/fdf134ea31dffeb7e43b97b2aeaf60f57cebb6293a7ec9024ce74fe83ae84eff: Zstandard compressed data (v0.8+), Dictionary ID: None
c9024ce74fe83ae84eff : 0 MB...     c9024ce74fe83ae84eff : Read error (39) : premature end 
+ set +x

+ git show --summary 9673ac4aac101ac016318e0f27592bfd585a24e6
commit 9673ac4aac101ac016318e0f27592bfd585a24e6
Author: Hubert <[email protected]>
Date:   Fri Sep 1 10:35:49 2023 +0200

    Add Shark actors for devnet actor bundle (#3434)

+ ls -l .git/lfs/objects/5a/9b/5a9b6a43425a42f0368109d043b71a8667889646b44073de4151c84e64c98716
-rw-r--r-- 1 aatif aatif 2433114 Sep  1 15:44 .git/lfs/objects/5a/9b/5a9b6a43425a42f0368109d043b71a8667889646b44073de4151c84e64c98716
+ file .git/lfs/objects/5a/9b/5a9b6a43425a42f0368109d043b71a8667889646b44073de4151c84e64c98716
.git/lfs/objects/5a/9b/5a9b6a43425a42f0368109d043b71a8667889646b44073de4151c84e64c98716: Zstandard compressed data (v0.8+), Dictionary ID: None
+ zstd --test .git/lfs/objects/5a/9b/5a9b6a43425a42f0368109d043b71a8667889646b44073de4151c84e64c98716
73de4151c84e64c98716 : 0 MB...     73de4151c84e64c98716 : Read error (39) : premature end 
+ set +x

@elmattic
Copy link
Contributor Author

Maybe we could add more checks in ensure_actor_bundle_includable function.

@elmattic
Copy link
Contributor Author

Looking at the file history, this has always been broken...
https://github.com/ChainSafe/forest/commits/main/assets/actor_bundles.car.zst

But we always managed to load bundle actors when starting the daemon... no?
Wondering if this has worked so far because the file is inlined thanks to include_bytes!.

@aatifsyed
Copy link
Contributor

Surprised nothing has been broken by this so far...

aatifsyed added a commit that referenced this issue Sep 13, 2023
aatifsyed added a commit that referenced this issue Sep 14, 2023
github-merge-queue bot pushed a commit that referenced this issue Sep 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants