Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use the justfile for CI builds #2730

Merged
merged 4 commits into from
Apr 18, 2024
Merged

Use the justfile for CI builds #2730

merged 4 commits into from
Apr 18, 2024

Conversation

paulcadman
Copy link
Collaborator

@paulcadman paulcadman commented Apr 16, 2024

This PR changes the CI build to use the justfile instead of the Makefile to run builds and tests. CI builds now take advantage of parallel module builds from #2729.

In order support this the runtime build target in the justfile now supports runtimeCcArg and runtimeLibtoolArg so that the CC and LIBTOOL Makefile argument can be set. This is required for the macOS build.

In addition this PR upgrades the stack setup step action. Previously the stack build flags included --fast which meant the whole project was rebuilt in the test step, this has also been fixed.

Overall this speeds up the CI:

  • Linux now takes 30mins (from 40mins)
  • macOS now takes 60mins (from 80mins)

@paulcadman paulcadman added the CI label Apr 16, 2024
@paulcadman paulcadman self-assigned this Apr 16, 2024
@paulcadman paulcadman added this to the 0.6.2 milestone Apr 16, 2024
@paulcadman paulcadman requested a review from janmasrovira April 16, 2024 14:33
@paulcadman paulcadman marked this pull request as ready for review April 16, 2024 14:33
@lukaszcz
Copy link
Collaborator

In addition this PR upgrades the stack setup step action. Previously the stack build flags included --fast which meant the whole project was rebuilt in the test step, this has also been fixed.

--fast is actually slow in this case, because the tests running time dwarfs compilation time and --fast turns off optimizations. I wonder how much of the improvement is due to running a faster test executable and how much due to parallelization of compilation.

Besides, we should test the release executable in the CI (compiled with optimizations).

@paulcadman
Copy link
Collaborator Author

paulcadman commented Apr 16, 2024

In addition this PR upgrades the stack setup step action. Previously the stack build flags included --fast which meant the whole project was rebuilt in the test step, this has also been fixed.

--fast is actually slow in this case, because the tests running time dwarfs compilation time and --fast turns off optimizations. I wonder how much of the improvement is due to running a faster test executable and how much due to parallelization of compilation.

Besides, we should test the release executable in the CI (compiled with optimizations).

This PR makes sure that the project is built only once. Previously we built the project twice, the first build (with the --fast flag) was never used.

Previously:

  • We first built the project with the --fast flag as part of the Stack setup step, and then rebuilt the project without the --fast flag in the test step

Now:

  • We build the project without the --fast flag in the Stack setup step, so the project does not get rebuilt in the test step. In particular the tests are run with the optimized build.

@paulcadman paulcadman force-pushed the ci-justfile branch 2 times, most recently from d9898d9 to 49c3192 Compare April 16, 2024 17:23
@janmasrovira janmasrovira merged commit b0fb240 into main Apr 18, 2024
4 checks passed
@janmasrovira janmasrovira deleted the ci-justfile branch April 18, 2024 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants