Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WARNING: Github runner disk layout is now a fast moving target, breaking this action for many #48

Open
easimon opened this issue Aug 6, 2024 · 2 comments

Comments

@easimon
Copy link
Owner

easimon commented Aug 6, 2024

Since the initial release I considered this action an ugly hack, to be used as a last resort only, because it depended on an undocumented "feature" of Github runners: an unused disk attached to the VM.

Over the last months, Github seems to regulary change the runner disk layout. The pattern is unclear for me. There seem to be differences between "free" runners and those used in Github Enterprise, but this does not seem to be everything.

Some runners now have both larger / and a larger /mnt, resulting in an enormous amount of total space, others have only 15GB free on / and seem to completely miss the temporary disk mounted on /mnt, so there's nothing to add space from.

Finding all these bits and pieces to make this work back when I created the action was quite fiddly -- but at least all "default" runners looked the same, so it was constant for everyone. Now, given this very mixed experience, I doubt that it makes sense to keep the action alive "as is" -- if at all.

Input on this is welcome, until then, this issue should serve as an additional warning to those considering the use of this action.

@easimon easimon pinned this issue Aug 6, 2024
@easimon easimon changed the title Github runner disk layout is now a fast moving target, breaking this action for many WARNING: Github runner disk layout is now a fast moving target, breaking this action for many Aug 6, 2024
@mirage335
Copy link

Please do continue maintaining this action. For builds which absolutely must have at minimum two simultaneous copies of a large file, or builds which do an analysis/diff of a package compared to previous builds, more disk space is essential.

In particular, it is very useful to run scheduled builds on the free runners. Many bugs are caught that otherwise would not be caught.

An extreme solution would be to use an LLM in the action to make reasoned decisions. While that may not seem reliable or feasible, in fact, I have had no trouble running 'ollama' with an 'augment' model of Llama 3.1 within GitHub Actions runners, and adding reasoning to shell scripts reliably with that. The bash shellcode to fetch that model is at https://github.com/mirage335-colossus/ubiquitous_bash/blob/ae067b11fd1bcaf1e72fd34dcd33a049caebbda1/ai/ollama/ollama.sh#L117 .

Another hint: seems runners for private runners are different, while public repository runners so far have remained unchanged for a long time for me.

@jianlins
Copy link

For some cases, /mnt is large enough for my use. Is there a way to mount that only, I guess it might avoid some of the errors?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants