Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

various fixes #1462

Merged
merged 6 commits into from
Jul 23, 2024
Merged

various fixes #1462

merged 6 commits into from
Jul 23, 2024

Conversation

Byron
Copy link
Member

@Byron Byron commented Jul 22, 2024

Tasks

Byron added 3 commits July 22, 2024 21:41
Let's not trade safety for some convenience.

This reduces performance by 5% at least (in the WebKit repository at 886077e077a496a6e398df52a4b7915d8cd68f76):

```
❯ hyperfine  "/Users/byron/dev/github.com/Byron/gitoxide/target/release/gix index entries ':(attr:export-ignore)' -i" "gix index entries ':(attr:export-ignore)' -i"
Benchmark 1: /Users/byron/dev/github.com/Byron/gitoxide/target/release/gix index entries ':(attr:export-ignore)' -i
  Time (mean ± σ):     820.1 ms ±  71.7 ms    [User: 759.9 ms, System: 85.9 ms]
  Range (min … max):   778.8 ms … 984.3 ms    10 runs

  Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.

Benchmark 2: gix index entries ':(attr:export-ignore)' -i
  Time (mean ± σ):     782.1 ms ±   1.8 ms    [User: 760.1 ms, System: 43.2 ms]
  Range (min … max):   780.0 ms … 786.0 ms    10 runs

Summary
  gix index entries ':(attr:export-ignore)' -i ran
    1.05 ± 0.09 times faster than /Users/byron/dev/github.com/Byron/gitoxide/target/release/gix index entries ':(attr:export-ignore)' -i
```
Byron added 3 commits July 23, 2024 14:40
This is quite typical in the wild where there ignore files are used
as include lists, instead of exclude-lists.
If they were, they would more easily be deleted by tooling like `gix clean`.
Copy link
Member Author

@Byron Byron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It took a while to write the tests, but once they were available homing on a fix was quite straightforward and felt nice, despite the complexity of the algorithm.

@@ -169,6 +169,7 @@ pub fn path(
.map(|platform| platform.excluded_kind())
})
.map_err(Error::ExcludesAccess)?
.filter(|_| filename_start_idx > 0)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line and the line below constitute the fix, it's quite neat.

The idea is that previously, the classifier didn't distinguish between the worktree root (and the contained repository) and sub-repositories. For the latter, the current classification is very much intentional, but the root should never ever be counted as ignored, and thus removable.

The commit with the fix also shows how the classification changes due to the fix, which is all it takes to prevent gix clean to go off the rail.

When looking for weaknesses, I also recommend using the pathspec, as it is quite integrated to the algorithm to have some (possibly unexpected) influence.

assert_eq!(
entries,
&[
entry_nokind(".git", Pruned).with_property(DotGit).with_match(Always),
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before the fix, this entry was ignored which was part of the problem. With the fix, it turned into a pruned DotGit entry, which naturally won't be touched by gix clean.

@Byron Byron marked this pull request as ready for review July 23, 2024 17:20
@Byron Byron merged commit b4dba1c into main Jul 23, 2024
19 checks passed
@Byron Byron deleted the fixes branch July 23, 2024 17:58
Copy link
Member

@EliahKagan EliahKagan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've checked manually on Windows 10 and Ubuntu 22.04 LTS, on both systems installing with cargo install --path ., that this fixes the cases covered in #1458. If I find other cases I'll open a new issue. Thanks!

@EliahKagan
Copy link
Member

Regarding the failure of the pure-rust-build job on CI from the merge commit, it passed in my fork, so maybe the failure would go away if the job is rerun.

@EliahKagan
Copy link
Member

maybe the failure would go away if the job is rerun.

I'm pleased to see it went away (and the subsequent commits, related to releasing, are also green).

If I find other cases I'll open a new issue.

I've opened #1464.

EliahKagan added a commit to EliahKagan/advisory-db that referenced this pull request Jul 24, 2024
gix-attributes was found by @ssbr to be unsound, as reported in
GitoxideLabs/gitoxide#1460. This adds an
informational notice for that, as discussed in comments there.

It looks like the affected code, having been introduced in
GitoxideLabs/gitoxide#400, was present in all
versions of the crate prior to the fix in 0.22.3 (which was one of
the bugs fixed in GitoxideLabs/gitoxide#1462).

Co-authored-by: Devin Jeanpierre <[email protected]>
Shnatsel pushed a commit to rustsec/advisory-db that referenced this pull request Jul 24, 2024
* Unsoundness notice for gix-attributes (kstring integration)

gix-attributes was found by @ssbr to be unsound, as reported in
GitoxideLabs/gitoxide#1460. This adds an
informational notice for that, as discussed in comments there.

It looks like the affected code, having been introduced in
GitoxideLabs/gitoxide#400, was present in all
versions of the crate prior to the fix in 0.22.3 (which was one of
the bugs fixed in GitoxideLabs/gitoxide#1462).

Co-authored-by: Devin Jeanpierre <[email protected]>

* Small adjustments for advisory

This makes some minor changes to the advisory description to adapt
the text from GitoxideLabs/gitoxide#1460 to be
an advisory. For the most part it has remained the same. Changes:

* Express the claim of unsoundness with more confidence, since it
  has been reviewed by the maintainer.

* Modify the link to the affected code to point to the latest tag
  for gix-attributes that has that code. The original link was to
  a branch, so it was broken when the fix was applied.

* Apply inline code formatting in a few more places, where doing
  so improves stylistic consistency.

---------

Co-authored-by: Devin Jeanpierre <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants