Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add aarch64-apple-darwin support #143

Closed
wants to merge 3 commits into from

Conversation

frol
Copy link
Contributor

@frol frol commented Feb 18, 2023

Resolves #133. cc @ashleygwilliams @Gankra

cargo-dist is now compiled for and also can be used to package other packages for aarch64-apple-darwin (M1/M2 chips)

I have tested this patch in a personal fork, here is the release: https://github.com/frol/cargo-dist/releases. Also, I have manually patched the CI scripts for my near-social CLI, and it worked like a charm (see that it now has aarch64-apple-darwin binary artifact), so I went ahead with creating this patch.

P.S. The release.yml file in main branch has a mix of CRLF and LF line endings. My guess is that git checks out ci.yml template with CRLF on Windows and then cargo-dist code generates some multiline strings with \n (push_github_artifacts_matrix_entry)

@frol
Copy link
Contributor Author

frol commented Feb 19, 2023

@Gankra I was also trying to understand the reasoning behind distribute_targets_to_runners. Having parallel compilation (through having a matrix group per build) is faster, and easier to reason about and debug, so there must be something behind this complication; what is that?

@Gankra
Copy link
Contributor

Gankra commented Feb 20, 2023

Doing a quick skim before I look at this properly tomorrow morning: this seems like a reasonable minimal approach, although I had been mentally planning for this to look more like:

  • teach cargo-dist to detect that you had "extra" tools like rustup/cross/strip/lipo/whatever (once in gather work, similar to makefile configure steps). Probably by running some-tool-i-want -V and storing the output to record what version we found.
  • if we find rustup, handle getting the toolchain installed for the user (I don't think we even need to bother asking for something as unobtrusive as a toolchain). This step failing can "just" be a warning unless we develop the capability to detect that e.g. a distro copy of rust has the relevant toolchains in its sysroot.

This would leave the CI untouched while also making local (non-CI) macos builds work better as well. It would also scale better to universal builds which need lipo (#77). Although the other platforms would however need to pull up a proper "cross-compile" core (#74).

cargo-dist is now compiled for and also can be used to package other packages for aarch64-apple-darwin (M1/M2 chips)
@frol frol force-pushed the feat/aarch64-apple-darwin-support branch from da01071 to 467b7dc Compare February 20, 2023 08:44
@frol
Copy link
Contributor Author

frol commented Feb 21, 2023

this seems like a reasonable minimal approach

Yes, this was exactly the goal of this PR, and I feel it solves the immediate pain point (#133) well without introducing complexity or diverging in a direction that will be hard to solve when a full-scale solution for cross-compilation will be developed.

M1/M2 Macbooks are quite popular among developers these days, and I would either need to come up with some clever logic in the installer that would pull x86_64 binary when aarch64 artifact is missing, or I could just build my CLI for aarch64-apple-darwin; the latter turned out to be quite a straightforward thing to do.

@Gankra Gankra added this to the 0.0.3 milestone Feb 21, 2023
@Gankra
Copy link
Contributor

Gankra commented Feb 21, 2023

I'm adding this as a tentative candidate for 0.0.3 which I'm going to cut in a couple days. If I find the bandwidth to do the Ideal solution I'll opt for that. Otherwise I'll land this to help people Get Stuff Done (well, assuming the implementation works).

My main concern is that the Ideal solution will restore the CI to its current state, so anyone who makes changes to their CI will have an extra round of otherwise-avoidable CI churn with this quick fix.

@frol
Copy link
Contributor Author

frol commented Feb 21, 2023

@Gankra Sorry, I don't see where you are heading. The changes to CI script are minimal and the current diff is weird since the original file had a mix of CRLF line endings and now it is completely CRLF (I feel it is a separate issue to address). Change the diff view to skip the whitespace changes to see the actual change.

- dist-args: --artifacts=local --target=x86_64-apple-darwin
+ dist-args: --artifacts=local --target=aarch64-apple-darwin --target=x86_64-apple-darwin

This is what essentially got changed, and I feel it is unavoidable unless you suggest cargo-dist should guess all the supported platforms and cross-compile to everything supported by default.

@Gankra
Copy link
Contributor

Gankra commented Feb 21, 2023

Ugh I guess I didn't properly fix #131 :(

But yes there will be some churn on CI as we refine things. There's definitely going to be a huge churn for 0.0.3 already, and that cost is already accepted. The question is after 0.0.3, when we ostensibly introduce the Ideal Solution.

  • if we don't accept this change then when we introduce the Ideal Solution, any users who adopt it will get that one line diff you noted -- not as likely to conflict, and easy to fix if it does.
  • if we do accept this change then users will essentially have to revert all the CI changes this PR makes, notably removing "targets" lines and the lines that configure rustup. this will be a lot more likely to conflict, and might mess up anyone who thought they could/should rely on "targets" being in the matrix.

It's not a big deal at all, I'm just noting the only real downside of accepting this PR.

(I have been pondering making the first task actually generating the matrix so there would be no churn even for the Ideal solution but that's a bigger change and makes it harder for users to configure anything...)

@frol
Copy link
Contributor Author

frol commented Feb 21, 2023

I believe the lines that setup rustup will inevitably get changed with the introduction of cross-compilation unless you want to go Zig-lang route and package cross-compilation toolkits for all the platforms or install Docker for users.

I may think of some way to avoid targets in the matrix if you don't want to give people a chance to depend on it, but so far I can only think of ugly hacks

@Gankra Gankra modified the milestones: 0.0.3, next Feb 23, 2023
@Gankra
Copy link
Contributor

Gankra commented Feb 23, 2023

We discussed this and 0.0.3 is too big already for us to be poking this feature too. I'm pushing this back to 0.0.4 which optimistically is "next week".

@Gankra
Copy link
Contributor

Gankra commented Mar 1, 2023

I implemented my version in #172, which was indeed thankfully straight-forward with my refactor done :)

This prerelease has it: https://github.com/axodotdev/cargo-dist/releases/tag/v0.0.4-prerelease.2

@Gankra Gankra closed this Mar 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cross compile aarch64-apple-darwin by default
2 participants