Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: truncated actor bundles #3485 #3488

Merged
merged 13 commits into from
Sep 20, 2023
Merged

fix: truncated actor bundles #3485 #3488

merged 13 commits into from
Sep 20, 2023

Conversation

aatifsyed
Copy link
Contributor

@aatifsyed aatifsyed requested a review from a team as a code owner September 13, 2023 15:23
@aatifsyed aatifsyed requested review from ruseinov and elmattic and removed request for a team September 13, 2023 15:23
@aatifsyed aatifsyed changed the title fix: actor bundles were incorrect fix: actor bundles are truncated #3485 Sep 13, 2023
src/utils/db/car_stream.rs Outdated Show resolved Hide resolved
@aatifsyed aatifsyed changed the title fix: actor bundles are truncated #3485 fix: truncated actor bundles #3485 Sep 13, 2023
@elmattic

This comment was marked as duplicate.

src/networks/mod.rs Outdated Show resolved Hide resolved
@aatifsyed
Copy link
Contributor Author

So the new actor bundle file is 12139433 bytes big? It's almost six time the previous size.

Not surprising - the uncompressed is 60MiB:

#!/usr/bin/env python3

import requests
import humanize

urls = [
    "https://github.com/filecoin-project/builtin-actors/releases/download/v9.0.3/builtin-actors-calibrationnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v10.0.0-rc.1/builtin-actors-calibrationnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v11.0.0-rc2/builtin-actors-calibrationnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v9.0.3/builtin-actors-devnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v10.0.0/builtin-actors-devnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v11.0.0/builtin-actors-devnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v9.0.3/builtin-actors-mainnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v10.0.0/builtin-actors-mainnet.car",
    "https://github.com/filecoin-project/builtin-actors/releases/download/v11.0.0/builtin-actors-mainnet.car",
]

total = 0

for url in urls:
    response = requests.head(url, allow_redirects=True)
    response.raise_for_status()
    content_length = response.headers["content-length"]
    total += int(content_length)

print(humanize.naturalsize(total, binary=True))
60.4 MiB

src/tool/subcommands/state_migration_cmd.rs Outdated Show resolved Hide resolved
src/tool/subcommands/state_migration_cmd.rs Outdated Show resolved Hide resolved
@aatifsyed aatifsyed force-pushed the aatifsyed/bad-actor-bundle branch from 284d871 to b9b1458 Compare September 14, 2023 23:03
Copy link
Contributor

@ruseinov ruseinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and way more readable.
Would be nice to have some details/comments about the car validation though.
it's not really clean why certain assumptions are enforced.

src/ipld/cid_hashset.rs Outdated Show resolved Hide resolved
generate_protobuf_code()
}

fn ensure_actor_bundle_includable() -> anyhow::Result<()> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we need this check anymore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's covered by the asset_integrity test now

@aatifsyed aatifsyed force-pushed the aatifsyed/bad-actor-bundle branch from 675957d to 4224ca3 Compare September 18, 2023 11:13
@lemmih lemmih marked this pull request as draft September 18, 2023 15:22
@aatifsyed aatifsyed force-pushed the aatifsyed/bad-actor-bundle branch from 4224ca3 to bf66624 Compare September 20, 2023 09:43
@aatifsyed aatifsyed marked this pull request as ready for review September 20, 2023 09:44
@aatifsyed aatifsyed requested a review from lemmih September 20, 2023 09:44
Copy link
Contributor

@lemmih lemmih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!

@aatifsyed aatifsyed enabled auto-merge September 20, 2023 11:10
@aatifsyed aatifsyed added this pull request to the merge queue Sep 20, 2023
Merged via the queue into main with commit 46b2958 Sep 20, 2023
@aatifsyed aatifsyed deleted the aatifsyed/bad-actor-bundle branch September 20, 2023 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants