Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sparse-checkout: avoid crash when switching between cone and non-cone #465

Merged
merged 3 commits into from
Nov 10, 2021
Merged

sparse-checkout: avoid crash when switching between cone and non-cone #465

merged 3 commits into from
Nov 10, 2021

Conversation

vdye
Copy link

@vdye vdye commented Nov 9, 2021

Background

The goal of these changes was to fix the infinite loop (and subsequent crash) that results from running the following commands in sequence in a repository:

$ git sparse-checkout init
$ git sparse-checkout add folder1/
$ git sparse-checkout init --cone
$ git sparse-checkout add folder2/

Changes

  • Disable cone mode when git sparse-checkout init --cone is called with existing non-cone patterns
  • Exit with an error when git sparse-checkout add is called in cone mode with existing non-cone patterns
  • Require starting '/' in cone mode patterns (otherwise interpreting them as invalid)

These patches, taken together, fix the failure case mentioned earlier (along with other, similar cases covered in the tests added to t1091).

Other Notes

I originally also had a commit that disallowed running add, set, list, and reapply when sparse-checkout is not enabled. It directly reversed the earlier decision to enable sparse-checkout when git sparse-checkout set is called with sparse-checkout disabled. As far as I can tell, it was added in V5 of the patch series introducing the command, but not much context is given as to why.

Most of the motivation to remove it came from the fact that 1) when sparse-checkout is disabled, the cone mode setting isn't preserved and 2) git sparse-checkout set doesn't include an option to set cone mode (or sparse index). Point 1) came up in the internal bug bash and led to some unexpected behavior; point 2) could partially resolve it, but I still find the (uncomfortably silent) auto-init to be unintuitive.

I ultimately didn't include the change here because lots of tests were relying on set to initialize the sparse-checkout (which, to me, indicates that users might as well?). I may introduce the change again (in a later pull request), but with some updates:

  • add, reapply, and list are still disallowed outside an active sparse-checkout
  • git sparse-checkout disable preserves the core.sparsecheckoutcone setting. It would be overwritten back to the default by git sparse-checkout init, but could be used when init-ing with git sparse-checkout set
  • set called in a disabled sparse checkout causes a warning message to be displayed that says 1) it's initializing the sparse checkout and 2) it's doing so with cone mode enabled/disabled, and to use git sparse-checkout init to change that setting

Let me know what you think - I definitely think there are improvements that could be made with the UX here (brought to light by the bug bash), but because I'm not really the target audience of git sparse-checkout, I'm not sure what the "right" UX is.

@vdye vdye requested a review from derrickstolee November 9, 2021 20:52
@vdye vdye self-assigned this Nov 9, 2021
@derrickstolee
Copy link

Changes

  • Disable cone mode when git sparse-checkout init --cone is called with existing non-cone patterns

I think this is fine, and hopefully there is a new warning.

Another approach would be to clear the patterns if they are not cone-mode, since we are doing an init. That's a bigger change to the expectations, though.

  • Exit with an error when git sparse-checkout add is called in cone mode with existing non-cone patterns

Good idea.

  • Require starting '/' in cone mode patterns (otherwise interpreting them as invalid)

I'm surprised this wasn't already there. Good find.

I originally also had a commit that disallowed running add, set, list, and reapply when sparse-checkout is not enabled. It directly reversed the earlier decision to enable sparse-checkout when git sparse-checkout set is called with sparse-checkout disabled. As far as I can tell, it was added in V5 of the patch series introducing the command, but not much context is given as to why.

I think this is something we can discuss upstream and could reverse that decision.

Most of the motivation to remove it came from the fact that 1) when sparse-checkout is disabled, the cone mode setting isn't preserved and 2) git sparse-checkout set doesn't include an option to set cone mode (or sparse index). Point 1) came up in the internal bug bash and led to some unexpected behavior; point 2) could partially resolve it, but I still find the (uncomfortably silent) auto-init to be unintuitive.

I ultimately didn't include the change here because lots of tests were relying on set to initialize the sparse-checkout (which, to me, indicates that users might as well?).

I doubt users are relying on that, but we would find out quickly. We also have some clear "experimental" flags on the builtin to clarify things like this. As far as I know, all documentation says to init first.

I may introduce the change again (in a later pull request), but with some updates:

  • add, reapply, and list are still disallowed outside an active sparse-checkout
  • git sparse-checkout disable preserves the core.sparsecheckoutcone setting. It would be overwritten back to the default by git sparse-checkout init, but could be used when init-ing with git sparse-checkout set
  • set called in a disabled sparse checkout causes a warning message to be displayed that says 1) it's initializing the sparse checkout and 2) it's doing so with cone mode enabled/disabled, and to use git sparse-checkout init to change that setting

Let me know what you think - I definitely think there are improvements that could be made with the UX here (brought to light by the bug bash), but because I'm not really the target audience of git sparse-checkout, I'm not sure what the "right" UX is.

These big questions are good ones to pursue outside of the release window. I'll dig in now.

Copy link

@derrickstolee derrickstolee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these changes make sense and look safe. Thanks for hardening the release!

Comment on lines +751 to +754
# NEEDSWORK: in the case of directory patterns like `deep/`, it might be worth trying
# to "correct" the patterns to match a cone mode style. However, that may be more difficult
# for nested directories (like `deep/deeper1/`) in which multiple individual patterns
# would be mapped from the original (`/deep/`, `!/deep/*/`, `/deep/deeper1/`).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an interesting idea, but I agree that the complexity might not be worth it.

I think "deep/" also means "include anything that's in a deep/ directory, including subdirectories such as /bin/deep/", so we can't really convert in this way without changing the definition. Even scanning the index for these deep/ directories is only enough for HEAD and not enough as HEAD moves throughout history.

@derrickstolee
Copy link

@vdye Sorry we didn't get these in before the -rc2 rebase. Can you rebase --onto the new tentative/vfs-2.34.0 branch (or git-for-windows/git:main if conflicts don't happen) then merge this?

vdye added 3 commits November 10, 2021 08:58
When reusing an existing `.git/info/sparse-checkout` file in `git
sparse-checkout init --cone`, warn user and fall back on non-cone mode
sparse checkout if the sparse patterns are not cone mode-compatible.  This
prevents users from ending up with erroneous/corrupted patterns in their
cone mode sparse-checkout (assuming no manual edits to the
`.git/info/sparse-checkout` file).

Additionally, by allowing the `git sparse-checkout init` to continue with
cone mode disabled, the user is able to correct or reset their patterns (via
`git sparse-checkout set`) and retry `git sparse-checkout init --cone`
without requiring manual editing of `.git/info/sparse-checkout`.

Signed-off-by: Victoria Dye <[email protected]>
Add check for `git sparse-checkout add` when existing patterns are not cone
mode-compatible, exiting when invalid patterns are found.

When a non-cone mode pattern (e.g., one containing glob pattern matching)
exists in the on-disk `.git/info/sparse-checkout` file, `git sparse-checkout
add` would warn that some patterns were incompatible with cone mode, but
otherwise succeed with exit code 0. However, the original (incompatible)
patterns would have been removed in the process. For example:

$ git sparse-checkout init
$ git sparse-checkout add folder*/
$ git config core.sparsecheckoutcone true
$ git sparse-checkout add different-folder/

would result in the patterns:

/*
!/*/
/different-folder/

To prevent the unintentional pattern removal, `git sparse-checkout add`
exits with an error if incompatible patterns were found (and therefore not
included in the hashmaps used to generate the output sparse pattern file).

Signed-off-by: Victoria Dye <[email protected]>
Add check for missing initial '/' when verifying cone patterns in
`add_pattern_to_hashsets(...)`, disabling cone patterns if not found. Update
tests in `t1091-sparse-checkout-builtin.sh` to ensure the starting '/' is
present unless the test verifies behavior when it is missing.

The implicit assumption of a starting '/' in cone mode patterns exists
throughout the code paths of `git sparse-checkout`, including in the
construction of the sparse pattern hashmaps. However, without enforcement of
that assumption, the presence of a directory pattern missing the starting
'/' would cause any subsequent cone mode `git sparse-checkout add` to enter
an infinite recursive loop in `insert_recursive_pattern(...)`
(`sparse-checkout.c`). By adding the starting '/' requirement to
`add_pattern_to_hashsets(...)` (`dir.c`), cone mode is disabled on the
pattern list, forcing `git sparse-checkout add` to exit early.

Signed-off-by: Victoria Dye <[email protected]>
@vdye vdye merged commit 7f113f1 into microsoft:tentative/vfs-2.34.0 Nov 10, 2021
@vdye vdye deleted the bugfix/sparse-checkout-fixes branch November 10, 2021 15:26
derrickstolee pushed a commit that referenced this pull request Nov 15, 2021
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 12, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 19, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 19, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 20, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 25, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
ldennington pushed a commit to ldennington/git that referenced this pull request Jan 25, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
dscho pushed a commit that referenced this pull request Feb 1, 2022
sparse-checkout: avoid crash when switching between cone and non-cone
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants