Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update agent formula to choose an m1/intel tarball based on the host computer #19

Merged
merged 2 commits into from
Dec 7, 2020

Conversation

yob
Copy link
Contributor

@yob yob commented Dec 3, 2020

Homebrew is working through the complexity of transitioning from Intel to Apple Silicon and much is up in the air.

However, our agent formula is quite simple. On Apple Silicon machines, we can ignore whether homebrew itself is running natively and just always download the Apple Silicon version of the agent.

On non Apple Silicon machines, we can ignore whether homebrew itself is running natively and just always download the Intel version of the agent.

This requires some helpers added to homebrew in Homebrew/brew#7995. Will they be available everywhere?

Before merging this we also need to consider the agent CD scripts that update this file automatically: https://github.com/buildkite/agent/blob/b2cf4e3d90f5b155505ec6eb53e007a4d88239d7/.buildkite/steps/release-homebrew.sh

…computer

Homebrew is working through the complexity of transitioning from Intel
to Apple Silicon and much is up in the air.

However, our agent formula is quite simple. On Apple Silicon machines,
we can ignore whether homebrew itself is running natively and just
always download the Apple Silicon version of the agent.

On non Apple Silicon machines, we can ignore whether homebrew itself is
running natively and just always download the Intel version of the
agent.

This requires some helpers added to homebrew in [1]. Will they be
available everywhere?

Before merging this we also need to consider the agent CD scripts that
update this file automatically [2].

[1] https://github.com/Homebrew/brew/pull/7995/files
[2] https://github.com/buildkite/agent/blob/b2cf4e3d90f5b155505ec6eb53e007a4d88239d7/.buildkite/steps/release-homebrew.sh
@yob yob requested a review from ticky December 3, 2020 14:37
@ticky
Copy link
Contributor

ticky commented Dec 3, 2020

I think we should follow what Homebrew is planning to do, which is installing based on the flavour of Homebrew installation, not the host CPU specifically.

It is expected that a user could have two Homebrew installations, one amd64 and one arm64, and that each must only install binaries of its flavour. I'm hoping to chase up the shape of this tooling as it becomes supported.

@ticky
Copy link
Contributor

ticky commented Dec 3, 2020

Looks like what we might want is Hardware::CPU.arm? instead.

@yob
Copy link
Contributor Author

yob commented Dec 3, 2020

I've been using macos and homebrew for less than 24 hours and you have somewhat closer connections than me, so we should 100% go with your suggestion 😂

@ticky
Copy link
Contributor

ticky commented Dec 3, 2020

Following this up with a hopefully-matching PR for the agent-release gear buildkite/agent#1346. Need to work out how to give that a run-through

@yob
Copy link
Contributor Author

yob commented Dec 4, 2020

I've tested this on a m1 mac and it works great.

First. I have brew installed to /opt/homebrew.

jh@Jamess-Air homebrew % cd $(brew --repo buildkite/buildkite)
jh@Jamess-Air homebrew-buildkite % git checkout  detect-arch
Branch 'detect-arch' set up to track remote branch 'detect-arch' from 'origin'.
Switched to a new branch 'detect-arch'

jh@Jamess-Air homebrew-buildkite % brew reinstall buildkite-agent
Warning: You are running macOS on a arm64 CPU architecture.
We do not provide support for this (yet).
Reinstall Homebrew under Rosetta 2 until we support it.
You will encounter build failures with some formulae.
Please create pull requests instead of asking for help on Homebrew's GitHub,
Twitter or any other official channels. You are responsible for resolving
any issues you experience while you are running this
unsupported configuration.

==> Downloading https://github.com/buildkite/agent/releases/download/v3.26.0/buildkite-agent-darwin-arm64-3.26.0.tar.gz
Already downloaded: /Users/jh/Library/Caches/Homebrew/downloads/2f0c5e21e2383b2c0b3d2c58a6ca143314937b15ff48d403d252270832210e11--buildkite-agent-darwin-arm64-3.26.0.tar.gz
==> Reinstalling buildkite/buildkite/buildkite-agent 
Ignoring existing config file at /opt/homebrew/etc/buildkite-agent/buildkite-agent.cfg
For changes see the updated dist copy at /opt/homebrew/Cellar/buildkite-agent/3.26.0/share/buildkite-agent/buildkite-agent.dist.cfg
==> Caveats
buildkite-agent is now installed!

Don't forget to update your configuration file with your agent token

Configuration file (to configure agent token, meta-data, priority, name, etc):
    /opt/homebrew/etc/buildkite-agent/buildkite-agent.cfg

Hooks directory (for customising the agent):
    /opt/homebrew/etc/buildkite-agent/hooks

Builds directory:
    /opt/homebrew/var/buildkite-agent/builds

Log paths:
    /opt/homebrew/var/log/buildkite-agent.log
    /opt/homebrew/var/log/buildkite-agent.error.log

If you set up the LaunchAgent, set your machine to auto-login as
your current user. It's also recommended to install Caffeine
(http://lightheadsw.com/caffeine/) to prevent your machine from going to
sleep or logging out.

To run multiple agents simply run the buildkite-agent start command
multiple times, or duplicate the LaunchAgent plist to create another
that starts on login.

To have launchd start buildkite/buildkite/buildkite-agent now and restart at login:
  brew services start buildkite/buildkite/buildkite-agent
Or, if you don't want/need a background service you can just run:
  buildkite-agent start
==> Summary
🍺  /opt/homebrew/Cellar/buildkite-agent/3.26.0: 5 files, 19.0MB, built in 1 second
jh@Jamess-Air homebrew-buildkite % lipo -archs $(which buildkite-agent)
arm64

I then installed a parallel version of homebrew to /usr/local (using these instructions):

% cat ~/.zshrc
export PATH=/opt/homebrew/bin:"${PATH}"
alias ibrew='arch -x86_64 /usr/local/bin/brew'

I then install the agent for ibrew:

jh@Jamess-Air homebrew-buildkite % pwd
/usr/local/Homebrew/Library/Taps/buildkite/homebrew-buildkite

% git checkout detect-arch
Branch 'detect-arch' set up to track remote branch 'detect-arch' from 'origin'.
Switched to a new branch 'detect-arch'

jh@Jamess-Air homebrew-buildkite % ibrew reinstall buildkite-agent
==> Downloading https://github.com/buildkite/agent/releases/download/v3.26.0/buildkite-agent-darwin-amd64-3.26.0.tar.gz
Already downloaded: /Users/jh/Library/Caches/Homebrew/downloads/d8f9cc08a5127cfccfff527ed77a8992ee5c1b51860f6d13f7d1a887b7e6ba1b--buildkite-agent-darwin-amd64-3.26.0.tar.gz
==> Reinstalling buildkite/buildkite/buildkite-agent 
Ignoring existing config file at /usr/local/etc/buildkite-agent/buildkite-agent.cfg
For changes see the updated dist copy at /usr/local/Cellar/buildkite-agent/3.26.0/share/buildkite-agent/buildkite-agent.dist.cfg
==> Caveats
buildkite-agent is now installed!

Don't forget to update your configuration file with your agent token

Configuration file (to configure agent token, meta-data, priority, name, etc):
    /usr/local/etc/buildkite-agent/buildkite-agent.cfg

Hooks directory (for customising the agent):
    /usr/local/etc/buildkite-agent/hooks

Builds directory:
    /usr/local/var/buildkite-agent/builds

Log paths:
    /usr/local/var/log/buildkite-agent.log
    /usr/local/var/log/buildkite-agent.error.log

If you set up the LaunchAgent, set your machine to auto-login as
your current user. It's also recommended to install Caffeine
(http://lightheadsw.com/caffeine/) to prevent your machine from going to
sleep or logging out.

To run multiple agents simply run the buildkite-agent start command
multiple times, or duplicate the LaunchAgent plist to create another
that starts on login.

To have launchd start buildkite/buildkite/buildkite-agent now and restart at login:
  brew services start buildkite/buildkite/buildkite-agent
Or, if you don't want/need a background service you can just run:
  buildkite-agent start
==> Summary
🍺  /usr/local/Cellar/buildkite-agent/3.26.0: 5 files, 34.7MB, built in 2 seconds

jh@Jamess-Air homebrew-buildkite % lipo -archs /usr/local/bin/buildkite-agent                      
x86_64

success!

@yob
Copy link
Contributor Author

yob commented Dec 5, 2020

@ticky do we need to consider whether Hardware::CPU.arm? is available on all users homebrew installs? Or will homebrew update itself and ensure that helpers like that are always available when the formula runs?

@ticky
Copy link
Contributor

ticky commented Dec 7, 2020

@ticky do we need to consider whether Hardware::CPU.arm? is available on all users homebrew installs? Or will homebrew update itself and ensure that helpers like that are always available when the formula runs?

Homebrew generally expects to be auto-updating both the code and the taps in lockstep, it’s possible to turn it off, but it turns both types of update off, so I think assuming the API will be there is entirely reasonable.

@yob
Copy link
Contributor Author

yob commented Dec 7, 2020

sweet 👍

@yob yob marked this pull request as ready for review December 7, 2020 22:31
@yob yob merged commit be80932 into master Dec 7, 2020
@yob yob deleted the detect-arch branch December 7, 2020 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants