Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

CI cache improvements #572

Merged
merged 7 commits into from
Aug 1, 2023
Merged

CI cache improvements #572

merged 7 commits into from
Aug 1, 2023

Conversation

neysofu
Copy link
Member

@neysofu neysofu commented Jul 26, 2023 •
edited
Loading

Description

  • Automatically removes caches associated with a branch, once its PR gets merged. This helps GHA's cache eviction algorithm to perform better.
  • Use mold to reduce compilation times.
  • Use cargo-nextest to run tests faster.
  • **Completely remove all usage of zsccache.** After some analysis of our CI logs, it looks like sccache` does not play well with GHA for projects as chunky as ours. It sends way too many cache restore requests, which result in 429s and slowdowns.
  • As an alternative to sccache, I'm doubling down on our usage of Swatinem/rust-cache. It now also caches examples/demo-prover compilation artifacts, and caches are shared between jobs when possible. Some more tweaks may be needed.

Linked Issues

Related to #188.

Testing

CI passes.

@neysofu neysofu force-pushed the filippo/ci-cache branch 3 times, most recently from 67ca87a to 3cb84e0 Compare July 28, 2023 13:03
@neysofu neysofu force-pushed the filippo/ci-cache branch from 330ffd5 to c4393d1 Compare July 28, 2023 16:05
@neysofu neysofu marked this pull request as ready for review July 28, 2023 16:41
@neysofu neysofu enabled auto-merge (squash) August 1, 2023 11:21
@neysofu neysofu disabled auto-merge August 1, 2023 14:34
@neysofu neysofu merged commit fcfc64a into nightly Aug 1, 2023
@neysofu neysofu deleted the filippo/ci-cache branch August 1, 2023 14:34
preston-evans98 pushed a commit that referenced this pull request Sep 14, 2023
@neysofu @preston-evans98
CI cache improvements (#572)
720bbe0 
* Improve CI

* Fix comments in rust.yml

* Improve CI docs and comments

* Fix Swatinem workspaces

* Disable cache writes on non-default branches
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@citizen-stig citizen-stig citizen-stig approved these changes

@bkolad bkolad bkolad approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@neysofu @citizen-stig @bkolad