exp apply: safeguard against unclean workspaces

[mattlbeck]

dvc exp apply will clear the workspace of untracked files and unstaged modifications. To prevent users losing work, it would be good to follow git's convention of preventing overwriting of untracked/modified files unless the user gives explicit permission.

• If dvc exp apply is executed on an unclean workspace, fail with a suggestion to stash work or specify a --force flag
• dvc exp apply --force acts like the current command does.

[daavoo]

Related: iterative/dvc.org#2397

[pmrowla]

This was the original behavior for exp apply, but after much discussion the conclusion was that a majority of typical DVC users would always be using exp apply --force. So the behavior was changed to force-apply by default (and the exp apply --no-force option was added for users who do not want the default behavior)

[mattlbeck]

@pmrowla I hadn't noticed the --no-force argument. On reflection, switching between experiments would necessitate use of a --force flag if it existed, and I guess that use-case is considered the most common.

I will monitor how I use exp apply for now, and whether I can easily encorporate --no-force into my workflow.

[dberenbaum]

@pmrowla Forcibly overwriting conflicts is intended as explained here and in iterative/dvc.org#2397, but it seems like experiments are overwriting everything in the workspace, even untracked files. This seems to conflict with iterative/dvc.org#2397 (comment).

Example:

$ dvc stage add -n mystage echo foo
Creating 'dvc.yaml'
Adding stage 'mystage' in 'dvc.yaml'

To track the changes with git, run:

    git add dvc.yaml

To enable auto staging, run:

        dvc config core.autostage true
$ dvc exp run
Running stage 'mystage':
> echo foo
foo
Generating lock file 'dvc.lock'
Updating lock file 'dvc.lock'

To track the changes with git, run:

    git add dvc.lock dvc.yaml

To enable auto staging, run:

        dvc config core.autostage true

Reproduced experiment(s): exp-396c5
Experiment results have been applied to your workspace.

To promote an experiment to a Git branch run:

        dvc exp branch <exp> <branch>

$ touch newfile
$ dvc exp apply exp-396c5
Changes for experiment 'exp-396c5' have been applied to your current workspace.
$ git status
On branch main
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        dvc.lock
        dvc.yaml

nothing added to commit but untracked files present (use "git add" to track)
$ ls
dvc.lock dvc.yaml

[dberenbaum]

Another user asked about this in discord: https://discord.com/channels/485586884165107732/485596304961962003/963800799220211782.

@pmrowla What do you think about bumping priority of this? It seems dangerous and potentially surprising to overwrite completely unrelated untracked files.

[pmrowla]

Unrelated untracked files being removed sounds like a bug. As for the priority, I guess I'm not sure if that should be opened as a separate issue from this one? It's unclear to me whether this issue is about the removing untracked files bug or if it's a higher level discussion about whether --force or --no-force should be the default behavior

[dberenbaum]

I guess I hijacked this ticket from @mattlbeck. As mentioned in #6930 (comment) and other comments above, we already have discussed the default behavior and it usually makes sense to leave the current defaults.

So let's either prioritize this issue to stop dropping unrelated untracked files, or close this one and open a new issue.