Remove CI Caching to preserve diskspace #2948
Conversation
| @@ -46,36 +46,29 @@ jobs: | |||
| # and thus do not depend on the OS, arch nor rust version. | |||
| path: /github/home/.cargo | |||
| key: cargo-cache- | |||
| - name: Cache Rust dependencies | |||
There was a problem hiding this comment.
I have never been convinced this caching is particularly effective so I am going to test what happens when I remove it...
There was a problem hiding this comment.
I wonder if something is wrong with the config 🤔
There was a problem hiding this comment.
I found this https://github.com/marketplace/actions/rust-cache which does perform some heuristics and cleaning.
There was a problem hiding this comment.
Maybe what's wrong here is that the .cargo die is not cached?
https://doc.rust-lang.org/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci
There was a problem hiding this comment.
This cache is automatically keyed by:
the github [job_id](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_id),
the rustc release / host / hash,
the value of some compiler-specific environment variables (eg. RUSTFLAGS, etc), and
a hash of all Cargo.lock / Cargo.toml files found anywhere in the repository (if present).
a hash of all rust-toolchain / rust-toolchain.toml files in the root of the repository (if present).
Seems particularly relevant to why the basic cache wasn't working for us
| @@ -107,12 +100,6 @@ jobs: | |||
| path: /github/home/.cargo | |||
| # this key equals the ones on `linux-build-lib` for re-use | |||
| key: cargo-cache- | |||
| - name: Cache Rust dependencies | |||
There was a problem hiding this comment.
For what it is worth the last time this CI check passed on master was https://github.com/apache/arrow-datafusion/runs/7399254770?check_suite_focus=true where it took 24m and 26 seconds (after the linux-build-lib job finished)
Codecov Report
@@ Coverage Diff @@
## master #2948 +/- ##
==========================================
- Coverage 85.44% 85.43% -0.01%
==========================================
Files 275 275
Lines 49690 49739 +49
==========================================
+ Hits 42458 42495 +37
- Misses 7232 7244 +12
|
| run: | | ||
| cargo build | ||
| cargo check |
|
Benchmark runs are scheduled for baseline = d11e28a and contender = b49093c. b49093c is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
Which issue does this PR close?
Closes #2947
Rationale for this change
CI is failing after running out of space; I am trying to reduce used space. I have also always suspected that the build caching is not particularly effective at reducing CI time
What changes are included in this PR?
targetdirectorybuildjob to usecargo check(rather than actually build binary output, just check compile errors)Are there any user-facing changes?
Not only does CI now pass, it is actually faster 😆
Here are the tests for the last time CI passed on master: https://github.com/apache/arrow-datafusion/runs/7399255003?check_suite_focus=true
Notably:
Here are the tests on this PR: https://github.com/apache/arrow-datafusion/runs/7435755729?check_suite_focus=true
Notably
There may well be other things going on that made the tests take longer on master / previously but I think this data demonstrates this code change doesn't make things noticeably worse