Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve CI build time #145

Merged
merged 1 commit into from
Nov 3, 2022

Conversation

chrisdoherty4
Copy link
Member

@chrisdoherty4 chrisdoherty4 commented Nov 3, 2022

The CI took in excess of 15m to run per commit. This was largely due to environment emulation used to build binaries and docker images for the supported platforms and serial execution of validation tools.

This change set does the following:

  • Cross compile binaries using the Go toolchain native support.
  • Perform multi platform builds using a matrix build that parallelizes the builds.
  • Run go test as a dedicated test job.
  • Cache Go dependencies using the setup-go v3 caching feature.
  • Updates all actions to their latest versions to ensure we don't begin failing due to various deprecations.

These changes results in an average build time reduction of roughly 600% from 19.5m to 3m (we should really think about using this setup across Tinkerbell).

To provide a sane local developer workflow the make image recipe has been updated to always build Linux binaries so it can be packaged into the final image. This means the binaries are no longer built in a container. The make build recipe is customizable using familiar GOOS and GOARCH variables defaulting to whatever your local platform is.

We may consider improving this in the future to limit version skew but given the reliability of the Go toolchain and Go being the only real dependency it seems OK to use the host environment for now. If more developers begin contributing we may need to normalize environments.

Minor bits

  • Removed the Drone CI related Makefile recipes.
  • Changed mergify to only require the images job given that job can't run without its pre-req jobs (all other jobs).

Closes #135

@chrisdoherty4 chrisdoherty4 changed the title Remove custom endpoints Crosscompile binaries instead of building in emulated environments Nov 3, 2022
@chrisdoherty4 chrisdoherty4 added the do-not-merge Signal to Mergify to block merging of the PR. label Nov 3, 2022
@codecov
Copy link

codecov bot commented Nov 3, 2022

Codecov Report

Merging #145 (9d420dc) into main (ae4d443) will decrease coverage by 3.50%.
The diff coverage is 100.00%.

❗ Current head 9d420dc differs from pull request most recent head 866aaa6. Consider uploading reports for the commit 866aaa6 to get more accurate results

@@            Coverage Diff             @@
##             main     #145      +/-   ##
==========================================
- Coverage   24.19%   20.68%   -3.51%     
==========================================
  Files           7        7              
  Lines         682      672      -10     
==========================================
- Hits          165      139      -26     
- Misses        494      514      +20     
+ Partials       23       19       -4     
Impacted Files Coverage Δ
internal/http/server.go 72.97% <100.00%> (-1.50%) ⬇️
internal/http/handlers.go 42.85% <0.00%> (-12.25%) ⬇️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@chrisdoherty4 chrisdoherty4 force-pushed the feat/crosscompile branch 6 times, most recently from face8ee to 1e688cc Compare November 3, 2022 15:05
@chrisdoherty4 chrisdoherty4 changed the title Crosscompile binaries instead of building in emulated environments Improve CI build time Nov 3, 2022
@chrisdoherty4 chrisdoherty4 requested review from tstromberg, displague and jacobweinstock and removed request for tstromberg November 3, 2022 15:26
@chrisdoherty4 chrisdoherty4 force-pushed the feat/crosscompile branch 2 times, most recently from 5257709 to a144f9e Compare November 3, 2022 15:36
@chrisdoherty4 chrisdoherty4 removed the do-not-merge Signal to Mergify to block merging of the PR. label Nov 3, 2022
@chrisdoherty4 chrisdoherty4 force-pushed the feat/crosscompile branch 3 times, most recently from da397ef to 3717f44 Compare November 3, 2022 16:22
@chrisdoherty4 chrisdoherty4 added the do-not-merge Signal to Mergify to block merging of the PR. label Nov 3, 2022
jacobweinstock
jacobweinstock previously approved these changes Nov 3, 2022
Instead of building binaries as part of the image build using QEMU we're
crosscompiling using Go's native support which should be faster.

Signed-off-by: Chris Doherty <[email protected]>
@jacobweinstock jacobweinstock added ready-to-merge Signal to Mergify to merge the PR. and removed do-not-merge Signal to Mergify to block merging of the PR. labels Nov 3, 2022
@jacobweinstock jacobweinstock removed the ready-to-merge Signal to Mergify to merge the PR. label Nov 3, 2022
@jacobweinstock jacobweinstock added the ready-to-merge Signal to Mergify to merge the PR. label Nov 3, 2022
@chrisdoherty4
Copy link
Member Author

@Mergifyio queue

@mergify
Copy link
Contributor

mergify bot commented Nov 3, 2022

Sorry but this command cannot run when the configuration is updated

@chrisdoherty4 chrisdoherty4 merged commit 271347c into tinkerbell:main Nov 3, 2022
@chrisdoherty4 chrisdoherty4 deleted the feat/crosscompile branch November 3, 2022 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-merge Signal to Mergify to merge the PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use go to cross compile for arm64
2 participants