Add ability to run with a snapshot of working directory

[cplee]

Currently act binds the local repo as /github/workspace in the containers for the actions. The problem is that any mutations that the actions make to the workspace affect the local repo.

We should add support for a new flag -t to instruct act to create a copy of the local repo in a new temp directory and bind that temp directory into the containers so that act can be run in a way that guarantees no changes are made on the local repo.

[Code0x58]

@cplee I think it would be simplest/best† from act's perspective if it always created a copy with git clone --recurse-submodules $local_repo_path, then running act would run the actions over exactly the same files as a hosted runner.

The temporary directory should be deleted (may have to use another container to delete it due to non-user ownership) unless act is told not to (for debugging), as your containers will probably leave things in there that can't be changed by the user (e.g. *.pyc files or any other artefacts made by actions, which may be owned by root/other).

Another thing would be that this temporary directory should only be acessable to root+the current user, otherwise you could be exposing secrets (e.g. on a shared test server).

† The files on your local system probably won't match what git gets for various reasons, e.g.

• filters need to be applied
• ignored files need to be excluded (although they may be committed already)
• non-comitted changes (staged+unstaged)
• submodules may or may not be checked out

[xendk]

While I think using a temp directory is a great idea as to not leave artifacts around, the other side of the coin is that running on the users files makes testing workflows easier. Also it would limit acts usefulness as a task runner. Not sure what I actually would prefer though.

[cplee]

@xendk agreed. My thought would be to leave the current behavior as the default, and then add a flag to use to activate this new behavior of creating a temp directory. thoughts?

[xendk]

@cplee One could make a case for the default to be the non-destructive option and put the current behavior behind a switch. A bit annoying for using it as a task runner, but there's something to be said for not touching the users checkout in the default case.

[Code0x58]

I think running in the original checkout may be useful if you aren't using filters (other than git-lfs) and want to do something something weird and wonderful(?) like only running some stages by hand - your task running? The thing I don't like about doing that by default is that it leaves room for surprises to casual users e.g. permissions on artefacts, "works on my machine".

I'm pro-fresh working directory by default as I am convinced it would make adoption smoother/simpler for people/teams. People getting caught out by "works on my machine" would be a pretty big detraction from the GitLab Actions CI/CD to me.

Would it cover your use case with something like:

rm -fr /tmp/my-workspace
act my-first-stage --use=/tmp/my-workspace # directory doesn't exist so make a fresh copy
act my-second-stage --use=/tmp/my-workspace # directory exists so just run action

[xendk]

What I'd suggest:

act

Runs in a new temporary directory which is deleted after use.

act -i

Runs in the original directory.

Personally I don't think I'd use a switch to specify a specific directory. To me it would suggest not removing the directory after use which I can't see why I would need that. But that's just my personal oppinion.

[Code0x58]

Could you describe your use case(s) for me? I imagined you were interested in running a subset of actions but it sounds like I misunderstand - particularly around "it would limit acts usefulness as a task runner".

I suppose for testing workflows you'd just want the workspace to not be deleted/cleaned. I suggest deleting them by default because as I imagine you'd be safest using a random temporary directory for each run to avoid conflicts, and that could fill up your disk - especially if you are running on a CI/CD machine.

The other reason I suggest deleting the temporary workspaces by default is for projects where there are contributors who aren't familiar with the issues from using docker, or even how GitHub actions work - I imagine this to be quite a large part of the population. I imagine there's a fair chance people would get caught on: temporary directories accumulating, temporary directories containing things owned by root thanks to Docker so they can't delete (or have to sudo or use another container with a mount), or non-clean state causing different outcomes. I suppose my motivation is to minimise unpleasant surprises for casual users, letting them use workflows with the minimum amount of thought as though they were just pushing to GitHub.

p.s. I imagine act --use=$PWD would have the same effect as your act -i. Whatever the conclusion is, you might find something like alias act='act -i'/alias act='act --use="$(pwd)"' handy

[xendk]

Could you describe your use case(s) for me?

I'm actually not using it as a task runner, I was merely taking the hint from the readme:

Local Task Runner - I love make. However, I also hate repeating myself. With act, you can use the GitHub Actions defined in your main.workflow file to replace your Makefile!

My use case for not using a temporary directory is working with the workflow itself. I was using act to test run the workflow before committing and pushing. In that case, creating a fresh clone would require me to commit every change.

I suppose my motivation is to minimise unpleasant surprises for casual users

I'll still contest that. Having act only run on committed changes will certainly trip up people used to make and similar tools. make will do something that incorporates your uncommitted changes, act wouldn't.

Running on a temp copy of your current checkout would be nice, but that involves a fair bit more of bitshuffling.

Actually, thinking about it and acts potential as a task runner, I'm more leaning towards keeping the current behavior and having act -c (for --clean) run in a temporary directory.

[Code0x58]

In makefiles that's whatever a user defines it as. In GitHub Actions the workspace gets thrown away at completion so it doesn't deal with the particulars of cleaning. How would act clean?

Personally I'm used to modifying commits locally before I push them to a remote branch with git commit --amend and other history changing commands. Squashing commits at merge time is another option.

I love the uses = <local path/git path/docker url> approach to things, and what that kind of thing lets you do, but in my mind act is a copy of the runners used in GitHub Actions, any additional use cases that you can have when you tweak the process locally is secondary. That may just be me.

I like the idea of catering for secondary use cases, that is why I don't think copy+run+delete is all that should be catered for, but I think the primary use is what should work smoothly out of the box. With what I'm suggesting you could still use makefiles perfectly easily to do things if you wanted to contain your tooling in Actions, so I don't think it compromises that vision.

[xendk]

In makefiles that's whatever a user defines it as.

Sure, but I'm yet to see a make build that requires you to commit your changes first.

How would act clean?

act clean? In make one does it explicitly, why not in act. But if you want it to do it, either cp -ar the original directory, or get fancy with overlay filesystems.

I'm quite familiar with git and history modification, but that's not the point. I don't commit to run make and i don't commit to run tests. Why should I commit to run act?

but I think the primary use is what should work smoothly out of the box

But what do you think the primary use is? It seems to be at odds with the stated purpose of the maintainer:

Fast Feedback - Rather than having to commit/push every time you want test out the changes you are making to your main.workflow file (or for any changes to embedded GitHub actions), you can use act to run the actions locally. The environment variables and filesystem are all configured to match what GitHub provides.
Local Task Runner - I love make. However, I also hate repeating myself. With act, you can use the GitHub Actions defined in your main.workflow file to replace your Makefile!

Which both points towards not using a clean slate.

Anyway, @cplee himself suggested adding a switch, and after some thinking, I think that's the best solution. Then you could alias github-actions to act -t.

[Code0x58]

I've seen quite a few makefiles that require commits (plenty in my current workplace), e.g. for things that ultimately end up using git describe to label a build or artefact, or use something like https://github.com/andrivet/ADVbumpversion which aborts if anything isn't committed.

It does come down to cplee's choice and vision for the project. I've covered my reasoning both for the copy approach by default, and ways to use that when like-for-like GitHub Action replication isn't the goal.

[sosedoff]

For those folks that are interested in the original proposal (to run act in the temp folder), i have a somewhat working implementation that i use myself quite often. When running act it'll create a new copy of the working dir (using dir copy, not git clone) and run the actions against it. So far, i found this approach to handle all the edge cases like uncommitted changes, temp files, etc without any modifications to the work directory, which i really like. Now, should this new behavior be a default one or triggered with a new flag? I'd go with default, but @cplee is to decide.

[cplee]

Thanks for all the feedback! I still like the option of keeping act closer to make in that it works on your local directory by default and then add a new flag to trigger working in a temp directory (with auto cleanup of temp directory on exit).

Next question...should the temp directory be populated from the current git worktree or from the latest git commit? My preference would be worktree. Thoughts?

[xendk]

Either works for me. I'm biased towards worktree as it allows for testing that it works on ones changes, but without any stray files one might have floating about.