Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: failed build in ARM #84

Merged
merged 13 commits into from
Jun 22, 2022
Merged

fix: failed build in ARM #84

merged 13 commits into from
Jun 22, 2022

Conversation

luislhl
Copy link
Contributor

@luislhl luislhl commented Jun 15, 2022

Motivation

We had problems building the cryptography package

See the error in https://github.com/HathorNetwork/tx-mining-service/runs/6905469491?check_suite_focus=true

Downgrading to alpine3.15 still errors out: https://github.com/HathorNetwork/tx-mining-service/runs/6907057307?check_suite_focus=true

Since the process apparently suffered a Kill -9, it's possible that it was using too much RAM during compilation and got OOM Killed.

Solution

See #84 (comment)

This is the working build after the fix: https://github.com/HathorNetwork/tx-mining-service/runs/6993659510?check_suite_focus=true

@luislhl luislhl changed the title Fix/rust build fix: failed build in ARM Jun 15, 2022
@luislhl luislhl self-assigned this Jun 20, 2022
@jansegre
Copy link
Member

I suggest switching the python base image from -alpine to -slim, which is Debian based. Mainly because it's easier and faster to just use the official wheels provided by cryptography, which use manylinux, which, if I'm not mistaken, works on Debian but not on Alpine.

I had a similar issue on hathor-core, however since we recently switched to a Debian base image it is now only affecting pypy+arm builds, because there aren't official wheels for those.

The other alternative I think would be to set-up our own runners, which we can make more beefy and also we can have arm runners that will run the arm tasks (although to make that work with the Docker build we'd have to make a few changes). But even just running our own runners seems like a big undertaking already.

@luislhl
Copy link
Contributor Author

luislhl commented Jun 21, 2022

I suggest switching the python base image from -alpine to -slim, which is Debian based. Mainly because it's easier and faster to just use the official wheels provided by cryptography, which use manylinux, which, if I'm not mistaken, works on Debian but not on Alpine.

Yeah, it makes sense to try to use the official wheels. But while learning about this, I saw that apparently they provide wheels for musllinux as well, not only manylinux: https://cryptography.io/en/latest/installation/#static-wheels

musllinux wheels should support Alpine.

Apparently our build was not using them just because Poetry was not prepared yet to use musllinux wheels: python-poetry/poetry#5225

After upgrading to Poetry 1.2.0b2, the build was able to use the wheels as expected.

So I think this is enough to solve it. What do you think?

@luislhl luislhl marked this pull request as ready for review June 21, 2022 21:57
@luislhl luislhl requested a review from msbrogli as a code owner June 21, 2022 21:57
@luislhl luislhl requested review from jansegre and pedroferreira1 and removed request for msbrogli June 21, 2022 21:58
Dockerfile Outdated Show resolved Hide resolved
.github/workflows/docker.yml Outdated Show resolved Hide resolved
.github/workflows/docker.yml Outdated Show resolved Hide resolved
@luislhl luislhl merged commit 3fd40f1 into dev Jun 22, 2022
@luislhl luislhl deleted the fix/rust-build branch June 22, 2022 17:18
@luislhl luislhl mentioned this pull request Jun 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants