[DO NOT MERGE YET] Rebase to v2.49.0 #729
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Calculate the number of symrefs, loose vs packed, and the maximal/accumulated length of local vs remote branches. Signed-off-by: Jeff Hostetler <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
With this commit, we gather statistics about the sizes of commits, trees, and blobs in the repository, and then present them in the form of "hexbins", i.e. log(16) histograms that show how many objects fall into the 0..15 bytes range, the 16..255 range, the 256..4095 range, etc. For commits, we also show the total count grouped by the number of parents, and for trees we additionally show the total count grouped by number of entries in the form of "qbins", i.e. log(4) histograms. Signed-off-by: Jeff Hostetler <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Create `struct large_item` and `struct large_item_vec` to capture the n largest commits, trees, and blobs under various scaling dimensions, such as size in bytes, number of commit parents, or number of entries in a tree. Each of these have a command line option to set them independently. Signed-off-by: Jeff Hostetler <[email protected]>
Add test case to demonstrate that `git index-pack -o <idx-path> pack-path` fails if <idx-path> does not end in ".idx" when `--rev-index` is enabled. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. The test here demonstrates the failure. (The test forces `--rev-index` to avoid interaction with `GIT_TEST_NO_WRITE_REV_INDEX` during CI runs.) Signed-off-by: Jeff Hostetler <[email protected]>
Include the pathname of each blob or tree in the large_item_vec to help identify the file or directory associated with the OID and size information. This pathname is computed during the path walk, so it reflects the first observed pathname seen for that OID during the traversal over all of the refs. Since the file or directory could have moved (without being modified), there may be multiple "correct" pathnames for a particular OID. Since we do not control the ref traversal order, we should consider it to be a "suggested pathname" for the OID. Signed-off-by: Jeff Hostetler <[email protected]>
Add a test verifying that sparse-checkout (with and without sparse index enabled) treat untracked files & directories correctly when changing sparse patterns. Specifically, it ensures that 'git sparse-checkout set' * deletes empty directories outside the sparse cone * does _not_ delete untracked files outside the sparse cone Signed-off-by: Victoria Dye <[email protected]>
Teach index-pack to silently omit the reverse index if the index file does not have the standard ".idx" suffix. In e37d0b8 (builtin/index-pack.c: write reverse indexes, 2021-01-25) we learned to create `.rev` reverse indexes in addition to `.idx` index files. The `.rev` file pathname is constructed by replacing the suffix on the `.idx` file. The code assumes a hard-coded "idx" suffix. In a8dd7e0 (config: enable `pack.writeReverseIndex` by default, 2023-04-12) reverse indexes were enabled by default. If the `-o <idx-path>` argument is used, the index file may have a different suffix. This causes an error when it tries to create the reverse index pathname. Since we do not know why the user requested a non-standard suffix for the index, we cannot guess what the proper corresponding suffix should be for the reverse index. So we disable it. The t5300 test has been updated to verify that we no longer error out and that the .rev file is not created. TODO We could warn the user that we skipped it (perhaps only if they TODO explicitly requested `--rev-index` on the command line). TODO TODO Ideally, we should add an `--rev-index-path=<path>` argument TODO or change `--rev-index` to take a pathname. TODO TODO I'll leave these questions for a future series. Signed-off-by: Jeff Hostetler <[email protected]>
Prefetch the value of GIT_TRACE2_DST_DEBUG during startup and before we try to open any Trace2 destination pathnames. Normally, Trace2 always silently fails if a destination target cannot be opened so that it doesn't affect the execution of a Git command. The command should run normally, but just not generate any trace data. This can make it difficult to debug a telemetry setup, since the user doesn't know why telemetry isn't being generated. If the environment variable GIT_TRACE2_DST_DEBUG is true, the Trace2 startup will print a warning message with the `errno` to make debugging easier. However, on Windows, looking up the env variable resets `errno` so the warning message always ends with `...tracing: No error` which is not very helpful. Prefetch the env variable at startup. This avoids the need to update each call-site to capture `errno` in the usual `saved-errno` variable. Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
This backports the `ds/advice-sparse-index-expansion` patches into `microsoft/git` which _just_ missed the v2.46.0 window. Signed-off-by: Johannes Schindelin <[email protected]>
Cherry-pick rev-index fixes from v2.41.0.vfs.0.5 into v2.42.0.*
Prefetch the value of GIT_TRACE2_DST_DEBUG during startup and before we try to open any Trace2 destination pathnames. Normally, Trace2 always silently fails if a destination target cannot be opened so that it doesn't affect the execution of a Git command. The command should run normally, but just not generate any trace data. This can make it difficult to debug a telemetry setup, since the user doesn't know why telemetry isn't being generated. If the environment variable GIT_TRACE2_DST_DEBUG is true, the Trace2 startup will print a warning message with the `errno` to make debugging easier. However, on Windows, looking up the env variable resets `errno` so the warning message always ends with `...tracing: No error` which is not very helpful. Prefetch the env variable at startup. This avoids the need to update each call-site to capture `errno` in the usual `saved-errno` variable.
Signed-off-by: Jeff Hostetler <[email protected]>
Computing `git name-rev` on each commit, tree, and blob in each of the various large_item_vec can be very expensive if there are too many refs, especially if the user doesn't need the result. Lets make it optional. The `--no-name-rev` option can save 50 calls to `git name-rev` since we have 5 large_item_vec's and each defaults to 10 items. Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
Signed-off-by: Jeff Hostetler <[email protected]>
While using the reset --stdin feature on windows path added may have a \r at the end of the path that wasn't getting removed so didn't match the path in the index and wasn't reset. Signed-off-by: Kevin Willford <[email protected]>
This topic branch brings in a new, experimental built-in command to assess the dimensions of a local repository. It is experimental and subject to change! It might grow new options, change its output, or even be moved into `git diagnose --analyze` or something like that. The hope is that this command, which was inspired by `git sizer` (https://github.com/github/git-sizer), will be helpful not only in diagnosing issues with large repositories, but also in modeling what shapes and sizes of repositories can be handled by Git (and as a corollary: where Git needs to improve to be able to accommodate the natural growth of repositories). Signed-off-by: Johannes Schindelin <[email protected]>
It has been a long-standing practice in Git for Windows to append `.windows.<n>`, and in microsoft/git to append `.vfs.0.0`. Let's keep doing that. Signed-off-by: Johannes Schindelin <[email protected]>
…sitories (#667) This command is inspired by [`git sizer`](https://github.com/github/git-sizer), having the advantage of being much closer to the internals of Git. The intention is to provide a built-in command that can be used to analyze large repositories for performance and scaling problems, for growth over time, and to correlate with other measurements (in particular with Trace2 data collected e.g. via https://github.com/git-ecosystem/trace2receiver/).
Since we really want to be based on a `.vfs.*` tag, let's make sure that there was a new-enough one, i.e. one that agrees with the first three version numbers of the recorded default version. This prevents e.g. v2.22.0.vfs.0.<some-huge-number>.<commit> from being used when the current release train was not yet tagged. It is important to get the first three numbers of the version right because e.g. Scalar makes decisions depending on those (such as assuming that the `git maintenance` built-in is not available, even though it actually _is_ available). Signed-off-by: Johannes Schindelin <[email protected]>
This header file will accumulate GVFS-specific definitions. Signed-off-by: Kevin Willford <[email protected]>
This does not do anything yet. The next patches will add various values for that config setting that correspond to the various features offered/required by GVFS. Signed-off-by: Kevin Willford <[email protected]> gvfs: refactor loading the core.gvfs config value This code change makes sure that the config value for core_gvfs is always loaded before checking it. Signed-off-by: Kevin Willford <[email protected]>
This takes a substantial amount of time, and if the user is reasonably sure that the files' integrity is not compromised, that time can be saved. Git no longer verifies the SHA-1 by default, anyway. Signed-off-by: Kevin Willford <[email protected]> Update for 2023-02-27: This feature was upstreamed as the index.skipHash config option. This resulted in some changes to the struct and some of the setup code. In particular, the config reading was moved to prepare_repo_settings(), so the core.gvfs bit check was moved there, too. Signed-off-by: Kevin Willford <[email protected]> Signed-off-by: Derrick Stolee <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Kevin Willford <[email protected]>
Prevent the sparse checkout to delete files that were marked with skip-worktree bit and are not in the sparse-checkout file. This is because everything with the skip-worktree bit turned on is being virtualized and will be removed with the change of HEAD. There was only one failing test when running with these changes that was checking to make sure the worktree narrows on checkout which was expected since we would no longer be narrowing the worktree. Update 2022-04-05: temporarily set 'sparse.expectfilesoutsideofpatterns' in test (until we start disabling the "remove present-despite-SKIP_WORKTREE" behavior with 'core.virtualfilesystem' in a later commit). Signed-off-by: Kevin Willford <[email protected]>
While performing a fetch with a virtual file system we know that there will be missing objects and we don't want to download them just because of the reachability of the commits. We also don't want to download a pack file with commits, trees, and blobs since these will be downloaded on demand. This flag will skip the first connectivity check and by returning zero will skip the upload pack. It will also skip the second connectivity check but continue to update the branches to the latest commit ids. Signed-off-by: Kevin Willford <[email protected]>
This adds hard-coded call to GVFS.hooks.exe before and after each Git command runs. To make sure that this is only called on repositories cloned with GVFS, we test for the tell-tale .gvfs. 2021-10-30: Recent movement of find_hook() to hook.c required moving these changes out of run-command.c to hook.c. Signed-off-by: Ben Peart <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
The clear_skip_worktree_from_present_files_sparse() method attempts to clear the skip worktree bit from cache entries in the index depending on when they exist in the workdir. When this comes across a sparse directory that actually exists in the workdir, then this method fails and signals that the index needs expansion. The index expansion already logs a reason, but this reason is separate from the path that caused this failure. Add logging to demonstrate this situation for full clarity. Signed-off-by: Derrick Stolee <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
Signed-off-by: Derrick Stolee <[email protected]>
Signed-off-by: Derrick Stolee <[email protected]>
I will intend to send this upstream after the 2.47.0 release cycle, but this should get to our microsoft/git users for maximum impact. Customers have been struggling with explaining why the sparse index expansion advice message is showing up. The advice to run 'git clean' has not always helped folks, and sometimes it is very unclear why we are running into trouble. These changes introduce a way to log a reason for the expansion into the trace2 logs so it can be found by requesting that a user enable tracing. While testing this, I created the most standard case that happens, which is to have an existing directory match a sparse directory in the index. In this case, it showed that two log messages were required. See the last commit for this new log message. Together, these two places show this kind of message in the `GIT_TRACE2_PERF` output (trimmed for clarity): ``` region_enter | index | label:clear_skip_worktree_from_present_files_sparse data | sparse-index | ..skip-worktree sparsedir:<my-sparse-path>/ data | index | ..sparse_path_count:362 data | index | ..sparse_lstat_count:732 region_leave | index | label:clear_skip_worktree_from_present_files_sparse data | sparse-index | expansion-reason:failed to clear skip-worktree while sparse ``` I added some tests to demonstrate that these logs are recorded, but it also seems difficult to hit some of these cases.
An internal customer reported a segfault when running `git sparse-checkout set` with the `index.sparse` config enabled. I was unable to reproduce it locally, but with their help we debugged into the failing process and discovered the following stacktrace: ``` #0 0x00007ff6318fb7b0 in rehash (map=0x3dfb00d0440, newsize=1048576) at hashmap.c:125 #1 0x00007ff6318fbc66 in hashmap_add (map=0x3dfb00d0440, entry=0x3dfb5c58bc8) at hashmap.c:247 #2 0x00007ff631937a70 in hash_index_entry (istate=0x3dfb00d0400, ce=0x3dfb5c58bc8) at name-hash.c:122 #3 0x00007ff631938a2f in add_name_hash (istate=0x3dfb00d0400, ce=0x3dfb5c58bc8) at name-hash.c:638 #4 0x00007ff631a064de in set_index_entry (istate=0x3dfb00d0400, nr=8291, ce=0x3dfb5c58bc8) at sparse-index.c:255 #5 0x00007ff631a06692 in add_path_to_index (oid=0x5ff130, base=0x5ff580, path=0x3dfb4b725da "<redacted>", mode=33188, context=0x5ff570) at sparse-index.c:307 #6 0x00007ff631a3b48c in read_tree_at (r=0x7ff631c026a0 <the_repo>, tree=0x3dfb5b41f60, base=0x5ff580, depth=2, pathspec=0x5ff5a0, fn=0x7ff631a064e5 <add_path_to_index>, context=0x5ff570) at tree.c:46 #7 0x00007ff631a3b60b in read_tree_at (r=0x7ff631c026a0 <the_repo>, tree=0x3dfb5b41e80, base=0x5ff580, depth=1, pathspec=0x5ff5a0, fn=0x7ff631a064e5 <add_path_to_index>, context=0x5ff570) at tree.c:80 #8 0x00007ff631a3b60b in read_tree_at (r=0x7ff631c026a0 <the_repo>, tree=0x3dfb5b41ac8, base=0x5ff580, depth=0, pathspec=0x5ff5a0, fn=0x7ff631a064e5 <add_path_to_index>, context=0x5ff570) at tree.c:80 #9 0x00007ff631a06a95 in expand_index (istate=0x3dfb00d0100, pl=0x0) at sparse-index.c:422 #10 0x00007ff631a06cbd in ensure_full_index (istate=0x3dfb00d0100) at sparse-index.c:456 #11 0x00007ff631990d08 in index_name_stage_pos (istate=0x3dfb00d0100, name=0x3dfb0020080 "algorithm/levenshtein", namelen=21, stage=0, search_mode=EXPAND_SPARSE) at read-cache.c:556 #12 0x00007ff631990d6c in index_name_pos (istate=0x3dfb00d0100, name=0x3dfb0020080 "algorithm/levenshtein", namelen=21) at read-cache.c:566 #13 0x00007ff63180dbb5 in sanitize_paths (argc=185, argv=0x3dfb0030018, prefix=0x0, skip_checks=0) at builtin/sparse-checkout.c:756 #14 0x00007ff63180de50 in sparse_checkout_set (argc=185, argv=0x3dfb0030018, prefix=0x0) at builtin/sparse-checkout.c:860 #15 0x00007ff63180e6c5 in cmd_sparse_checkout (argc=186, argv=0x3dfb0030018, prefix=0x0) at builtin/sparse-checkout.c:1063 #16 0x00007ff6317234cb in run_builtin (p=0x7ff631ad9b38 <commands+2808>, argc=187, argv=0x3dfb0030018) at git.c:548 #17 0x00007ff6317239c0 in handle_builtin (argc=187, argv=0x3dfb0030018) at git.c:808 #18 0x00007ff631723c7d in run_argv (argcp=0x5ffdd0, argv=0x5ffd78) at git.c:877 #19 0x00007ff6317241d1 in cmd_main (argc=187, argv=0x3dfb0030018) at git.c:1017 #20 0x00007ff631838b60 in main (argc=190, argv=0x3dfb0030000) at common-main.c:64 ``` The very bottom of the stack being the `rehash()` method from `hashmap.c` as called within the `name-hash` API made me look at where these hashmaps were being used in the sparse index logic. These were being copied across indexes, which seems dangerous. Indeed, clearing these hashmaps and setting them as not initialized fixes the segfault. The second commit is a response to a test failure that happens in `t1092-sparse-checkout-compatibility.sh` where `git stash pop` starts to fail because the underlying `git checkout-index` process fails due to colliding files. Passing the `-f` flag appears to work, but it's unclear why this name-hash change causes that change in behavior.
In preparation for allowing both the --shallow and --path-walk options in the 'git pack-objects' builtin, create a new 'edge_aggressive' option in the path-walk API. This option will help walk the boundary more thoroughly and help avoid sending extra objects during fetches and pushes. The only use of the 'edge_hint_aggressive' option in the revision API is within mark_edges_uninteresting(), which is usually called before between prepare_revision_walk() and before visiting commits with get_revision(). In prepare_revision_walk(), the UNINTERESTING commits are walked until a boundary is found. We didn't use this in the past because we would mark objects UNINTERESTING after doing the initial commit walk to the boundary. While we should be marking these objects as UNINTERESTING, we shouldn't _emit_ them all via the path-walk algorithm or else our delta calculations will get really slow. Based on these observations, the way we were handling the UNINTERESTING flag in walk_objects_by_path() was overly complicated and buggy. A lot of it can be removed and simplified to work with this new approach. It also means that we will see the UNINTERESTING boundaries of paths when doing a default path-walk call, changing some existing test cases. Signed-off-by: Derrick Stolee <[email protected]>
In some instances (particularly the `read_object` hook), the `cmd` attribute is set to an `strdup()`ed value. This value needs to be released in the end! Since other users assign a non-`strdup()`ed value, be careful to add _another_ attribute (called `to_free`) that can hold a reference to such a string that needs to be released once the sub process is done. Signed-off-by: Johannes Schindelin <[email protected]>
There does not appear to be anything particularly incompatible about the --shallow and --path-walk options of 'git pack-objects'. If shallow commits are to be handled differently, then it is by the revision walk that defines the commit set and which are interesting or uninteresting. However, before the previous change, a trivial removal of the warning would cause a failure in t5500-fetch-pack.sh when GIT_TEST_PACK_PATH_WALK is enabled. The shallow fetch would provide more objects than we desired, due to some incorrect behavior of the path-walk API, especially around walking uninteresting objects. To also cover the symmetrical case of pushing from a shallow clone, add a new test to t5538-push-shallow.sh that confirms the correct behavior of pushing only the new object. This works to validate both the --path-walk and --no-path-walk case when toggling the GIT_TEST_PACK_PATH_WALK environment variable. This test would have failed in the --path-walk case if we created it before the previous change. Signed-off-by: Derrick Stolee <[email protected]>
This fixes a leak that is not detected by Git's test suite (but by microsoft/git's). Signed-off-by: Johannes Schindelin <[email protected]>
It can be notoriously difficult to detect if delta bases are being computed properly during 'git push'. Construct an example where it will make a kilobyte worth of difference when a delta base is not found. We can then use the progress indicators to distinguish between bytes and KiB depending on whether the delta base is found and used. Signed-off-by: Derrick Stolee <[email protected]>
This fixes a leak that is not detected by Git's own test suite (but by microsoft/git's, in the t9210-scalar.sh test). Signed-off-by: Johannes Schindelin <[email protected]>
This pull request aims to correct a pretty big issue when dealing with UNINTERESTING objects in the path-walk API. They somehow were only exposed when trying to perform a push from a shallow clone. This will require rewriting the upstream version so this is avoided from the start, but we can do a forward fix for now. The key issue is that the path-walk API was not walking UNINTERESTING trees at the right time, and the way it was being done was more complicated than it needed to be. This changes some of the way the path-walk API works in the presence of UNINTERSTING commits, but these are good changes to make. I had briefly attempted to remove the use of the `edge_aggressive` option in `struct path_walk_info` in favor of using the `--objects-edge-aggressive` option in the revision struct. When I started down that road, though, I somehow got myself into a bind of things not working correctly. I backed out to this version that is working with our test cases. I tested this using the thin and big pack tests in `p5313` which had the same performance as before this change. The new change is that in a shallow clone we can get the same `git push` improvements. I was hung up on testing this for a long time as I wasn't getting the same results in my shallow clone as in my regular clones. It turns out that I had forgotten to use `--no-reuse-delta` in my test command, so it was picking the deltas that were given by the initial clone instead of picking new ones per the algorithm. 🤦🏻
The check for dubious ownership has one particular quirk on Windows: if running as an administrator, files owned by the Administrators _group_ are considered owned by the user. The rationale for that is: When running in elevated mode, Git creates files that aren't owned by the individual user but by the Administrators group. There is yet another quirk, though: The check I introduced to determine whether the current user is an administrator uses the `CheckTokenMembership()` function with the current process token. And that check only succeeds when running in elevated mode! Let's be a bit more lenient here and look harder whether the current user is an administrator. We do this by looking for a so-called "linked token". That token exists when administrators run in non-elevated mode, and can be used to create a new process in elevated mode. And feeding _that_ token to the `CheckTokenMembership()` function succeeds! Signed-off-by: Johannes Schindelin <[email protected]>
This adds a new sub-sub-command for `test-tool`, simply passing through the command-line arguments to the `is_path_owned_by_current_user()` function. Signed-off-by: Johannes Schindelin <[email protected]>
Git v2.48.0 has become even more stringent about leaks. Signed-off-by: Johannes Schindelin <[email protected]>
The --path-walk option in `git pack-objects` is implied by the
pack.usePathWalk=true config value. This is intended to help the
packfile generation within `git push` specifically.
While this config does enable the path-walk feature, it does not lead to
the expected levels of compression in the cases it was designed to
handle. This is due to the default implication of the --reuse-delta
option as well as auto-GC.
In the performance tests used to evaluate the --path-walk option, such
as those in p5313, the --no-reuse-delta option is used to ensure that
deltas are recomputed according to the new object walk. However, it was
assumed (I assumed this) that when the objects were loose from
client-side operations that better deltas would be computed during this
operation. This wasn't confirmed because the test process used data that
was fetched from real repositories and thus existed in packed form only.
I was able to confirm that this does not reproduce when the objects to
push are loose. Careful use of making the pushed commit unreachable and
loosening the objects via `git repack -Ad` helps to confirm my
suspicions here. Independent of this change, I'm pushing for these
pipeline agents to set `gc.auto=0` before creating their Git objects. In
the current setup, the repo is adding objects and then incrementally
repacking them and ending up with bad cross-path deltas. This approach
can help scenarios where that makes sense, but will not cover all of our
users without them choosing to opt-in to background maintenance (and
even then, an incremental repack could cost them efficiency).
In order to make sure we are getting the intended compression in `git
push`, this change enforces the spawned `git pack-objects` process to
use `--no-reuse-delta`.
As far as I can tell, the main motivation for implying the --reuse-delta
option by default is two-fold:
1. The code in send-pack.c that executes 'git pack-objects' is ignorant
of whether the current process is a client pushing to a remote or a
remote sending a fetch or clone to a client.
2. For servers, it is critical that they trust the previously computed
deltas whenever possible, or they could overload their CPU
resources.
There's also the side that most servers use repacking logic that will
replace any bad deltas that are sent by clients (or at least, that's the
hope; we've seen that repacks can also pick bad deltas).
This commit also adds a test case that demonstrates that `git -c
pack.usePathWalk=true push` now avoids reusing deltas.
To do this, the test case constructs a pack with a horrendously
inefficient delta object, then verifies that the pack on the receiving
side of the `push` fails to have such an inefficient delta.
The test case would probably be a lot more readable if hex numbers were
used instead of octal numbers, but alas, `printf "\x<hex>"` is not
portable, only `printf "\<octal>"` is. For example, dash's built-in
`printf` function simply prints `\x` verbatim while bash's built-in
happily converts this construct to the corresponding byte.
Signed-off-by: Derrick Stolee <[email protected]>
Signed-off-by: Johannes Schindelin <[email protected]>
The check for dubious ownership has one particular quirk on Windows: if running as an administrator, files owned by the Administrators _group_ are considered owned by the user. The rationale for that is: When running in elevated mode, Git creates files that aren't owned by the individual user but by the Administrators group. There is yet another quirk, though: The check I introduced to determine whether the current user is an administrator uses the `CheckTokenMembership()` function with the current process token. And that check only succeeds when running in elevated mode! Let's be a bit more lenient here and look harder whether the current user is an administrator. We do this by looking for a so-called "linked token". That token exists when administrators run in non-elevated mode, and can be used to create a new process in elevated mode. And feeding _that_ token to the `CheckTokenMembership()` function succeeds!
Tests in t7900 assume the state of the `maintenance.strategy` config setting; set/unset by previous tests. Correct this by explictly unsetting and re-setting the config at the start of the tests. Signed-off-by: Matthew John Cheetham <[email protected]>
Update the WinGet release workflow to match the updating manifest in `microsoft/winget-pkgs`, where there are now four installation options: - x86_64 / x64 with machine scope - x86_64 / x64 with user scope - aarch64 / arm64 with machine scope - aarch64 / arm64 with user scope Signed-off-by: Matthew John Cheetham <[email protected]>
The --path-walk option in 'git pack-objects' is implied by the
pack.usePathWalk=true config value. This is intended to help the
packfile generation within 'git push' specifically.
While this config does enable the path-walk feature, it does not lead
the expected levels of compression in the cases it was designed to
handle. This is due to the default implication of the --reuse-delta
option as well as auto-GC.
In the performance tests used to evaluate the --path-walk option, such
as those in p5313, the --no-reuse-delta option is used to ensure that
deltas are recomputed according to the new object walk. However, it was
assumed (I assumed this) that when the objects were loose from
client-side operations that better deltas would be computed during this
operation. This wasn't confirmed because the test process used data that
was fetched from real repositories and thus existed in packed form only.
I was able to confirm that this does not reproduce when the objects to
push are loose. Careful use of making the pushed commit unreachable and
loosening the objects via 'git repack -Ad' helps to confirm my
suspicions here. Independent of this change, I'm pushing for these
pipeline agents to set 'gc.auto=0' before creating their Git objects. In
the current setup, the repo is adding objects and then incrementally
repacking them and ending up with bad cross-path deltas. This approach
can help scenarios where that makes sense, but will not cover all of our
users without them choosing to opt-in to background maintenance (and
even then, an incremental repack could cost them efficiency).
In order to make sure we are getting the intended compression in 'git
push', this change makes the --path-walk option imply --no-reuse-delta
when the --reuse-delta option is not provided.
As far as I can tell, the main motivation for implying the --reuse-delta
option by default is two-fold:
1. The code in send-pack.c that executes 'git pack-objects' is ignorant
of whether the current process is a client pushing to a remote or a
remote sending a fetch or clone to a client.
2. For servers, it is critical that they trust the previously computed
deltas whenever possible, or they could overload their CPU resources.
There's also the side that most servers use repacking logic that will
replace any bad deltas that are sent by clients (or at least, that's the
hope; we've seen that repacks can also pick bad deltas).
The --path-walk option at the moment is not compatible with reachability
bitmaps, so is not planned to be used by Git servers. Thus, we can
reasonably assume (for now) that the --path-walk option is assuming a
client-side scenario, either a push or a repack. The repack option will
be explicit about the --reuse-delta option or not.
One thing to be careful about is background maintenance, which uses a
list of objects instead of refs, so we condition this on the case where
the --path-walk option will be effective by checking that the --revs
option was provided.
Alternative options considered included:
* Adding _another_ config ('pack.reuseDelta=false') to opt-in to this
choice. However, we already have pack.usePathWalk=true as an opt-in to
"do the right thing to make my data small" as far as our internal users
are concerned.
* Modify the chain between builtin/push.c, transport.c, and
builtin/send-pack.c to communicate that we are in "push" mode, not
within a fetch or clone. However, this seemed like overkill. It may be
beneficial in the future to pass through a mode like this, but it does
not meet the bar for the immediate need.
Reviewers, please see git-for-windows#5171 for the baseline
implementation of this feature within Git for Windows and thus
microsoft/git. This feature is still under review upstream.
Introduce a new maintenance task, `cache-local-objects`, that operates on Scalar or VFS for Git repositories with a per-volume, shared object cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose objects from the repository object directory to the shared cache. Older versions of `microsoft/git` incorrectly placed packfiles in the repository object directory instead of the shared cache; this task will help clean up existing clones impacted by that issue. Migration of packfiles involves the following steps for each pack: 1. Hardlink (or copy): a. the .pack file b. the .keep file c. the .rev file 2. Move (or copy + delete) the .idx file 3. Delete/unlink: a. the .pack file b. the .keep file c. the .rev file Moving the index file after the others ensures the pack is not read from the new cache directory until all associated files (rev, keep) exist in the cache directory also. Moving loose objects operates as a move, or copy + delete. Signed-off-by: Matthew John Cheetham <[email protected]>
Add the `cache-local-objects` maintenance task to the list of tasks run by the `scalar run` command. It's often easier for users to run the shorter `scalar run` command than the equivalent `git maintenance` command. Signed-off-by: Matthew John Cheetham <[email protected]>
Introduce a new maintenance task, `cache-local-objects`, that operates on Scalar or VFS for Git repositories with a per-volume, shared object cache (specified by `gvfs.sharedCache`) to migrate packfiles and loose objects from the repository object directory to the shared cache. Older versions of `microsoft/git` incorrectly placed packfiles in the repository object directory instead of the shared cache; this task will help clean up existing clones impacted by that issue. Fixes #716
@derrickstolee when I reverted this, the test still passed. Since the patch caused merge conflicts anyway because there was no longer any special handling needed to force off the This now looks a lot simpler, so much so that I have my doubts, even if t5538 still succeeds. @derrickstolee does this look correct? |
I forgot that `VN` can be empty... Signed-off-by: Johannes Schindelin <[email protected]>
Oh right, I forgot to adjust the file extension after resolving the merge conflicts... _sigh_ Signed-off-by: Johannes Schindelin <[email protected]>
…nsions ... and another `.txt` -> `.adoc` fall-out. Signed-off-by: Johannes Schindelin <[email protected]>
Upstream Git decided that `-h` should all of a sudden go to `stdout` instead of `stderr`. Signed-off-by: Johannes Schindelin <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Range-diff relative to vfs-2.48.1
2: 132ad6a = 1: 9d523d0 sparse-index.c: fix use of index hashes in expand_index
5: 2c03380 = 2: f5655d0 t: remove advice from some tests
1: 84ba3bb = 3: 9b5a803 survey: calculate more stats on refs
4: 3bd3d96 = 4: c65a454 survey: show some commits/trees/blobs histograms
9: a6bdfa4 ! 5: dc23c7b survey: add vector of largest objects for various scaling dimensions
@@ Commit message Signed-off-by: Jeff Hostetler <[email protected]> - ## Documentation/config/survey.txt ## -@@ Documentation/config/survey.txt: survey.*:: + ## Documentation/config/survey.adoc ## +@@ Documentation/config/survey.adoc: survey.*:: top:: This integer value implies `--top=<N>`, specifying the number of entries in the detail tables. @@ Documentation/config/survey.txt: survey.*:: + linkgit:git-survey[1]. -- - ## Documentation/git-survey.txt ## -@@ Documentation/git-survey.txt: only refs for the given options are added. + ## Documentation/git-survey.adoc ## +@@ Documentation/git-survey.adoc: only refs for the given options are added. --other:: Add notes (`refs/notes/`) and stashes (`refs/stash/`) to the set. @@ Documentation/git-survey.txt: only refs for the given options are added. OUTPUT ------ -@@ Documentation/git-survey.txt: Reachable Object Summary +@@ Documentation/git-survey.adoc: Reachable Object Summary The reachable object summary shows the total number of each kind of Git object, including tags, commits, trees, and blobs.3: 6309b70 = 6: f976222 t5300: confirm failure of git index-pack when non-idx suffix requested
10: 32189b9 = 7: 9895728 survey: add pathname of blob or tree to large_item_vec
6: 6a53749 = 8: 034fdfc t1092: add test for untracked files and directories
7: 2a08f0a = 9: d60b546 index-pack: disable rev-index if index file has non .idx suffix
8: be2f825 = 10: 50a6aa7 trace2: prefetch value of GIT_TRACE2_DST_DEBUG at startup
11: 9399a12 = 11: fd05a52 survey: add commit-oid to large_item detail
12: c8be55d ! 12: 82fc24d survey: add commit name-rev lookup to each large_item
@@ builtin/survey.c: static void do_load_refs(struct survey_context *ctx, +{ + if (ctx->opts.show_progress) { + ctx->progress_total = 0; -+ ctx->progress = start_progress(_("Resolving name-revs..."), 0); ++ ctx->progress = start_progress(ctx->repo, _("Resolving name-revs..."), 0); + } + + large_item_vec_lookup_name_rev(ctx, ctx->report.reachable_objects.commits.vec_largest_by_nr_parents);13: f36fb84 ! 13: b531833 survey: add --no-name-rev option
@@ Commit message Signed-off-by: Jeff Hostetler <[email protected]> - ## Documentation/config/survey.txt ## -@@ Documentation/config/survey.txt: survey.*:: + ## Documentation/config/survey.adoc ## +@@ Documentation/config/survey.adoc: survey.*:: background with these options. + -- @@ Documentation/config/survey.txt: survey.*:: This boolean value implies the `--[no-]verbose` option. progress:: - ## Documentation/git-survey.txt ## -@@ Documentation/git-survey.txt: OPTIONS + ## Documentation/git-survey.adoc ## +@@ Documentation/git-survey.adoc: OPTIONS --progress:: Show progress. This is automatically enabled when interactive. @@ builtin/survey.c: static void large_item_vec_lookup_name_rev(struct survey_conte + if (ctx->opts.show_progress) { ctx->progress_total = 0; - ctx->progress = start_progress(_("Resolving name-revs..."), 0); + ctx->progress = start_progress(ctx->repo, _("Resolving name-revs..."), 0); @@ builtin/survey.c: static void survey_phase_objects(struct survey_context *ctx) release_revisions(&revs); trace2_region_leave("survey", "phase/objects", ctx->repo);14: 5d11a62 = 14: 10ee868 survey: started TODO list at bottom of source file
15: 0c02488 = 15: f77f27c survey: expanded TODO list at the bottom of the source file
16: 7758c74 = 16: 30fa141 survey: expanded TODO with more notes
17: 2785521 = 17: 6f0b2ff reset --stdin: trim carriage return from the paths
18: 82a9afa ! 18: 8c8f12e Identify microsoft/git via a distinct version suffix
19: 9d182fd < -: ------------ gvfs: ensure that the version is based on a GVFS tag
-: ------------ > 19: b8a595f gvfs: ensure that the version is based on a GVFS tag
20: 33a8988 = 20: bfd4c30 gvfs: add a GVFS-specific header file
21: e781a50 ! 21: 33ef9e7 gvfs: add the core.gvfs config setting
@@ Commit message Signed-off-by: Kevin Willford <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.multiPackIndex:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.multiPackIndex:: single index. See linkgit:git-multi-pack-index[1] for more information. Defaults to true.22: 22583f7 ! 22: 1d55ba9 gvfs: add the feature to skip writing the index' SHA-1
@@ Commit message Signed-off-by: Derrick Stolee <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.multiPackIndex:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.multiPackIndex:: information. Defaults to true. core.gvfs::23: eca6d35 ! 23: 3ef7191 gvfs: add the feature that blobs may be missing
@@ Commit message Signed-off-by: Kevin Willford <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.gvfs:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.gvfs:: GVFS_SKIP_SHA_ON_INDEX:: Bit value 1 Disables the calculation of the sha when writing the index24: ad33b6a ! 24: 07ebfc1 gvfs: prevent files to be deleted outside the sparse checkout
@@ Commit message Signed-off-by: Kevin Willford <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.gvfs:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.gvfs:: Bit value 4 Normally git write-tree ensures that the objects referenced by the directory exist in the object database. This option disables this check.25: dc7f44e ! 25: 786d556 gvfs: optionally skip reachability checks/upload pack during fetch
@@ Commit message Signed-off-by: Kevin Willford <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.gvfs:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.gvfs:: directory. This will allow virtualized working directories to detect the change to HEAD and use the new commit tree to show the files that are in the working directory.26: d13ff90 ! 26: 9044574 gvfs: ensure all filters and EOL conversions are blocked
@@ Commit message Signed-off-by: Ben Peart <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.gvfs:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.gvfs:: since these will be downloaded on demand. This flag will skip the checks on the reachability of objects during a fetch as well as the upload pack so that extraneous objects don't get downloaded.27: 767a6ba = 27: 3cf5d75 gvfs: allow "virtualizing" objects
28: d0f04ac = 28: 11e6e50 Hydrate missing loose objects in check_and_freshen()
29: d5b9f91 = 29: 85b7cc6 sha1_file: when writing objects, skip the read_object_hook
30: bf15ebc = 30: 8649964 gvfs: add global command pre and post hook procs
31: f3b6bd1 = 31: e1fb608 t0400: verify that the hook is called correctly from a subdirectory
32: 506300a = 32: 25f7cf5 Pass PID of git process to hooks.
33: b5e57ab = 33: b1d4ea7 pre-command: always respect core.hooksPath
34: aacfa5b = 34: 735b36e sparse-checkout: update files with a modify/delete conflict
35: 51da332 = 35: e8f10fc sparse-checkout: avoid writing entries with the skip-worktree bit
36: 5357567 = 36: d88679e Do not remove files outside the sparse-checkout
37: 60e7946 ! 37: 144d10d send-pack: do not check for sha1 file when GVFS_MISSING_OK set
38: bf32c09 = 38: 0f29f7c cache-tree: remove use of strbuf_addf in update_one
39: d66a1dc = 39: dd3c97d gvfs: block unsupported commands when running in a GVFS repo
40: c918116 = 40: 2a24741 worktree: allow in Scalar repositories
41: b62a93d = 41: de7459b gvfs: allow overriding core.gvfs
42: 4bde6ad = 42: 9da480c BRANCHES.md: Add explanation of branches and using forks
43: 225a9aa ! 43: 651739e Add virtual file system settings and hook proc
@@ Commit message Signed-off-by: Ben Peart <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: Version 2 uses an opaque string so that the monitor can return + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: Version 2 uses an opaque string so that the monitor can return something that can be used to determine what files have changed without race conditions. @@ Documentation/config/core.txt: Version 2 uses an opaque string so that the monit If false, the ctime differences between the index and the working tree are ignored; useful when the inode change time - ## Documentation/githooks.txt ## -@@ Documentation/githooks.txt: and "0" meaning they were not. + ## Documentation/githooks.adoc ## +@@ Documentation/githooks.adoc: and "0" meaning they were not. Only one parameter should be set to "1" when the hook runs. The hook running passing "1", "1" should not be possible.44: dafd0c9 = 44: 76162a4 virtualfilesystem: don't run the virtual file system hook if the index has been redirected
45: c127664 = 45: eb3bc52 virtualfilesystem: check if directory is included
46: caddbc7 = 46: 86decdf backwards-compatibility: support the post-indexchanged hook
47: bd95866 = 47: db2f4f9 gvfs: verify that the built-in FSMonitor is disabled
48: e47db22 = 48: 8ed0a4d wt-status: add trace2 data for sparse-checkout percentage
49: 7ee6b9c = 49: a2a143c wt-status: add VFS hydration percentage to normal
git statusoutput50: 6d18e4d ! 50: ae0e051 status: add status serialization mechanism
@@ Commit message Signed-off-by: Jeff Hostetler <[email protected]> Signed-off-by: Derrick Stolee <[email protected]> - ## Documentation/config/status.txt ## -@@ Documentation/config/status.txt: status.submoduleSummary:: + ## Documentation/config/status.adoc ## +@@ Documentation/config/status.adoc: status.submoduleSummary:: the --ignore-submodules=dirty command-line option or the 'git submodule summary' command, which shows a similar output but does not honor these settings. @@ Documentation/config/status.txt: status.submoduleSummary:: + `--deserialize=<path>` on the command line. If the cache file is + invalid or stale, git will fall-back and compute status normally. - ## Documentation/git-status.txt ## -@@ Documentation/git-status.txt: ignored, then the directory is not shown, but all contents are shown. + ## Documentation/git-status.adoc ## +@@ Documentation/git-status.adoc: ignored, then the directory is not shown, but all contents are shown. threshold. See also linkgit:git-diff[1] `--find-renames`. @@ Documentation/git-status.txt: ignored, then the directory is not shown, but all <pathspec>...:: See the 'pathspec' entry in linkgit:gitglossary[7]. -@@ Documentation/git-status.txt: quoted as explained for the configuration variable `core.quotePath` +@@ Documentation/git-status.adoc: quoted as explained for the configuration variable `core.quotePath` (see linkgit:git-config[1]).51: 049317d = 51: dbe4ecb Teach ahead-behind and serialized status to play nicely together
52: dce73bd ! 52: a8cbfc0 status: serialize to path
@@ Commit message Signed-off-by: Jeff Hostetler <[email protected]> - ## Documentation/git-status.txt ## -@@ Documentation/git-status.txt: ignored, then the directory is not shown, but all contents are shown. + ## Documentation/git-status.adoc ## +@@ Documentation/git-status.adoc: ignored, then the directory is not shown, but all contents are shown. threshold. See also linkgit:git-diff[1] `--find-renames`.53: d621a28 = 53: 47650b4 status: reject deserialize in V2 and conflicts
54: a55ed71 = 54: f99b6ba serialize-status: serialize global and repo-local exclude file metadata
55: 602ba37 ! 55: 3976042 status: deserialization wait
@@ Commit message Signed-off-by: Jeff Hostetler <[email protected]> - ## Documentation/config/status.txt ## -@@ Documentation/config/status.txt: status.deserializePath:: + ## Documentation/config/status.adoc ## +@@ Documentation/config/status.adoc: status.deserializePath:: generated by `--serialize`. This will be overridden by `--deserialize=<path>` on the command line. If the cache file is invalid or stale, git will fall-back and compute status normally.56: 4d09641 = 56: 468ade8 merge-recursive: avoid confusing logic in was_dirty()
57: d1af813 = 57: 545f3b7 merge-recursive: add some defensive coding to was_dirty()
58: 6ca188e = 58: 371cca9 merge-recursive: teach was_dirty() about the virtualfilesystem
59: e4da5b0 = 59: 2c4bf11 status: deserialize with -uno does not print correct hint
60: 39943b3 = 60: 46faf82 fsmonitor: check CE_FSMONITOR_VALID in ce_uptodate
61: 5398a74 = 61: 5f3048e fsmonitor: add script for debugging and update script for tests
62: 3e1eb2b ! 62: 5e16288 status: disable deserialize when verbose output requested.
@@ builtin/commit.c +#include "trace2.h" static const char * const builtin_commit_usage[] = { - N_("git commit [-a | --interactive | --patch] [-s] [-v] [-u<mode>] [--amend]\n" + N_("git commit [-a | --interactive | --patch] [-s] [-v] [-u[<mode>]] [--amend]\n" @@ builtin/commit.c: struct repository *repo UNUSED) */ try_deserialize = (!do_serialize &&63: 05b0721 = 63: 9fc6141 t7524: add test for verbose status deserialzation
64: 665e1b4 = 64: a467a5c deserialize-status: silently fallback if we cannot read cache file
65: 207b578 = 65: 60430a2 gvfs:trace2:data: add trace2 tracing around read_object_process
66: 6ce1dc2 = 66: 4e5b659 gvfs:trace2:data: status deserialization information
67: 875d110 = 67: 1337743 gvfs:trace2:data: status serialization
68: 450ff3e = 68: 5968dbc gvfs:trace2:data: add vfs stats
69: 2854a6d = 69: 7addf6d trace2: refactor setting process starting time
70: 1736ae9 = 70: 8d68658 trace2:gvfs:experiment: clear_ce_flags_1
71: e425e7b = 71: 9db7e71 trace2:gvfs:experiment: report_tracking
72: 7e0b32f = 72: 52b9c2d trace2:gvfs:experiment: read_cache: annotate thread usage in read-cache
73: c5285f0 ! 73: 43cfe2f trace2:gvfs:experiment: read-cache: time read/write of cache-tree extension
@@ read-cache.c: static int read_index_extension(struct index_state *istate, + trace2_region_leave("index", "read/extension/cache_tree", NULL); break; case CACHE_EXT_RESOLVE_UNDO: - istate->resolve_undo = resolve_undo_read(data, sz); + istate->resolve_undo = resolve_undo_read(data, sz, the_hash_algo); @@ read-cache.c: static int do_write_index(struct index_state *istate, struct tempfile *tempfile, !drop_cache_tree && istate->cache_tree) { strbuf_reset(&sb);74: 0ed8665 = 74: 6b31f85 trace2:gvfs:experiment: add region to apply_virtualfilesystem()
75: b8fef43 = 75: 7e46fbb trace2:gvfs:experiment: add region around unpack_trees()
76: b31529b = 76: e43eef6 trace2:gvfs:experiment: add region to cache_tree_fully_valid()
77: aff9def = 77: e8818ff trace2:gvfs:experiment: add unpack_entry() counter to unpack_trees() and report_tracking()
78: a5e0b36 = 78: a6d8adf trace2:gvfs:experiment: increase default event depth for unpack-tree data
79: 3ccceea = 79: ef54200 trace2:gvfs:experiment: add data for check_updates() in unpack_trees()
80: 2be5e4d = 80: 7691c42 Trace2:gvfs:experiment: capture more 'tracking' details
81: c50cb77 = 81: c817d5b credential: set trace2_child_class for credential manager children
82: dcf71fa = 82: 4e874f6 sub-process: do not borrow cmd pointer from caller
83: 166067a = 83: 1a2c45d sub-process: add subprocess_start_argv()
84: 2f3b474 = 84: 9bc4c4b sha1-file: add function to update existing loose object cache
85: 345431b = 85: 33ea560 packfile: add install_packed_git_and_mru()
86: 0a0f5f0 = 86: 359b994 index-pack: avoid immediate object fetch while parsing packfile
87: 317d0a7 ! 87: 43e771c gvfs-helper: create tool to fetch objects using the GVFS Protocol
88: f046226 = 88: 7d3f1ac sha1-file: create shared-cache directory if it doesn't exist
89: a90d8ca = 89: ef71673 gvfs-helper: better handling of network errors
90: c3d896a = 90: b6aa270 gvfs-helper-client: properly update loose cache with fetched OID
91: 55b1c71 ! 91: df92c79 gvfs-helper: V2 robust retry and throttling
92: a22ab61 = 92: 36acf3d gvfs-helper: expose gvfs/objects GET and POST semantics
93: 7a54bac = 93: 643db1f gvfs-helper: dramatically reduce progress noise
94: 50ca17c = 94: c595614 gvfs-helper-client.h: define struct object_id
95: f394fd7 = 95: 6adb693 gvfs-helper: handle pack-file after single POST request
96: d3a0365 ! 96: 3ee3fc4 test-gvfs-prococol, t5799: tests for gvfs-helper
@@ gvfs-helper.c: static void do_throttle_spin(struct gh__request_params *params, strbuf_addstr(®ion, gh__server_type_label[params->server_type]); trace2_region_enter("gvfs-helper", region.buf, NULL); -- progress = start_progress(progress_msg, duration); +- progress = start_progress(the_repository, progress_msg, duration); + if (gh__cmd_opts.show_progress) -+ progress = start_progress(progress_msg, duration); ++ progress = start_progress(the_repository, progress_msg, duration); + while (now < end) { display_progress(progress, (now - begin)); @@ t/helper/test-gvfs-protocol.c (new) + unsigned char compressed[4096]; + git_zstream stream; + struct object_id oid_check; -+ git_hash_ctx c; ++ struct git_hash_ctx c; + int object_header_len; + int ret; +97: 2f81d39 = 97: c672d48 gvfs-helper: move result-list construction into install functions
98: 21e7f37 ! 98: af382c2 t5799: add support for POST to return either a loose object or packfile
@@ t/helper/test-gvfs-protocol.c: static enum worker_result do__gvfs_config__get(st { #define MAX_HEADER_LEN 32 @@ t/helper/test-gvfs-protocol.c: static enum worker_result send_loose_object(const struct object_info *oi, - git_hash_ctx c; + struct git_hash_ctx c; int object_header_len; int ret; + unsigned flags = 0;99: d87f0c7 = 99: 4c04ad4 t5799: cleanup wc-l and grep-c lines
100: 45e6315 = 100: 39f5436 gvfs-helper: verify loose objects after write
101: 24a1c1d = 101: 017157a t7599: create corrupt blob test
102: ecb818e ! 102: ecd2def gvfs-helper: add prefetch support
103: 1909ee1 = 103: 4cd1164 gvfs-helper: add prefetch .keep file for last packfile
104: e92cd75 = 104: 85ee633 gvfs-helper: do one read in my_copy_fd_len_tail()
105: afdd8ea = 105: 3bc5997 gvfs-helper: move content-type warning for prefetch packs
106: 41e4d0c ! 106: dabc10d fetch: use gvfs-helper prefetch under config
@@ Commit message Signed-off-by: Derrick Stolee <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.gvfs:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.gvfs:: is first accessed and brought down to the client. Git.exe can't currently tell the first access vs subsequent accesses so this flag just blocks them from occurring at all.107: 9bb58b4 = 107: cd741af gvfs-helper: better support for concurrent packfile fetches
108: c5a4a8f = 108: e418e1c remote-curl: do not call fetch-pack when using gvfs-helper
109: 8d2fdbb = 109: d64ec45 fetch: reprepare packs before checking connectivity
110: 7c8e98a = 110: 75dbc7a gvfs-helper: retry when creating temp files
111: a0c0c62 = 111: 373c4da sparse: avoid warnings about known cURL issues in gvfs-helper.c
112: d1efc18 = 112: cdb259c gvfs-helper: add --max-retries to prefetch verb
113: 8aa590a = 113: a1fe2d7 t5799: add tests to detect corrupt pack/idx files in prefetch
114: e64962e = 114: 4fb49fe gvfs-helper: ignore .idx files in prefetch multi-part responses
115: 90b1c78 = 115: b5708a3 t5799: explicitly test gvfs-helper --fallback and --no-fallback
116: c70e49f ! 116: fa58d65 gvfs-helper: don't fallback with new config
@@ Commit message Signed-off-by: Derrick Stolee <[email protected]> - ## Documentation/config/gvfs.txt ## -@@ Documentation/config/gvfs.txt: gvfs.cache-server:: + ## Documentation/config/gvfs.adoc ## +@@ Documentation/config/gvfs.adoc: gvfs.cache-server:: gvfs.sharedcache:: TODO117: 07e0f90 = 117: 6b85031 test-gvfs-protocol: add cache_http_503 to mayhem
119: 920d3f7 = 118: c1c6bd0 maintenance: care about gvfs.sharedCache config
120: b6749e0 = 119: 3ab168b unpack-trees:virtualfilesystem: Improve efficiency of clear_ce_flags
118: 0598e5f = 120: 8068ebb t5799: add unit tests for new
gvfs.fallbackconfig setting121: 7592c76 = 121: 9287163 homebrew: add GitHub workflow to release Cask
122: 7e56c23 ! 122: 37ebca3 Adding winget workflows
123: 34700f6 = 123: 5e8c67e Disable the
monitor-componentsworkflow in msft-git124: bc62204 = 124: 8688653 .github: enable windows builds on microsoft fork
125: 716a315 ! 125: c75d92b release: create initial Windows installer build workflow
126: 43b4d91 = 126: 07b396e help: special-case HOST_CPU
universal127: a9af4ad ! 127: f0dcb8f release: add Mac OSX installer build
@@ .github/macos-installer/Makefile (new) + touch $@ + +$(BUILD_DIR)/git-$(VERSION)/osx-built-subtree: ++ $(SUBMAKE) -C $(BUILD_DIR)/git-$(VERSION)/Documentation asciidoc.conf + cd $(BUILD_DIR)/git-$(VERSION)/contrib/subtree; $(SUBMAKE) XML_CATALOG_FILES="$(XML_CATALOG_FILES)" all git-subtree.1 + touch $@ +128: f6400e6 ! 128: 465c1d1 release: build unsigned Ubuntu .deb package
129: 4ebc418 = 129: e77a36d release: add signing step for .deb package
130: 1f99600 = 130: 33ebe9a release: create draft GitHub release with packages & installers
131: 07f6052 ! 131: db67113 build-git-installers: publish gpg public key
132: 8c886c0 = 132: a619fd5 release: continue pestering until user upgrades
133: 4e30bb0 = 133: b355c33 dist: archive HEAD instead of HEAD^{tree}
134: 8a24a5f = 134: e0744c3 release: include GIT_BUILT_FROM_COMMIT in MacOS build
135: 7363256 = 135: 7e6d8e2 release: add installer validation
136: bea6180 ! 136: 9749a7a update-microsoft-git: create barebones builtin
137: c567faa = 137: cb73013 update-microsoft-git: Windows implementation
138: 13f763c = 138: afe6032 update-microsoft-git: use brew on macOS
139: 512dfbd = 139: 5e8b393 .github: update ISSUE_TEMPLATE.md for microsoft/git
140: 6a8250f = 140: 24cdaf1 .github: update PULL_REQUEST_TEMPLATE.md
141: 676c07e = 141: 6179b75 Adjust README.md for microsoft/git
142: 1526f65 ! 142: e64212e git_config_set_multivar_in_file_gently(): add a lock timeout
@@ Commit message Signed-off-by: Johannes Schindelin <[email protected]> - ## Documentation/config/core.txt ## -@@ Documentation/config/core.txt: core.WSLCompat:: + ## Documentation/config/core.adoc ## +@@ Documentation/config/core.adoc: core.WSLCompat:: The default value is false. When set to true, Git will set the mode bits of the file in the way of wsl, so that the executable flag of files can be set or read correctly.143: 469b549 = 143: f2d3394 scalar: set the config write-lock timeout to 150ms
144: 4a81210 = 144: b117668 scalar: add docs from microsoft/scalar
145: 298b29b = 145: 8ad629c scalar (Windows): use forward slashes as directory separators
146: 9ad768c = 146: 0a8c7b0 scalar: add retry logic to run_git()
147: 4a09510 = 147: 4a64d97 scalar: support the
configcommand for backwards compatibility148: 5a5e189 = 148: 1c19253 scalar: implement a minimal JSON parser
149: 2e80737 ! 149: 0388217 scalar clone: support GVFS-enabled remote repositories
150: 49102f6 = 150: 1f20613 test-gvfs-protocol: also serve smart protocol
151: 1fa4f46 = 151: 7110c95 gvfs-helper: add the
endpointcommand152: a8a4485 = 152: 8b9b557 dir_inside_of(): handle directory separators correctly
153: 0f4bc44 = 153: ddc0f56 scalar: disable authentication in unattended mode
154: c270115 ! 154: 686f7b0 scalar: do initialize
gvfs.sharedCache@@ Commit message Signed-off-by: Johannes Schindelin <[email protected]> - ## Documentation/scalar.txt ## -@@ Documentation/scalar.txt: SYNOPSIS + ## Documentation/scalar.adoc ## +@@ Documentation/scalar.adoc: SYNOPSIS -------- [verse] scalar clone [--single-branch] [--branch <main-branch>] [--full-clone] @@ Documentation/scalar.txt: SYNOPSIS scalar list scalar register [<enlistment>] scalar unregister [<enlistment>] -@@ Documentation/scalar.txt: cloning. If the HEAD at the remote did not point at any branch when +@@ Documentation/scalar.adoc: cloning. If the HEAD at the remote did not point at any branch when A sparse-checkout is initialized by default. This behavior can be turned off via `--full-clone`. @@ Documentation/scalar.txt: cloning. If the HEAD at the remote did not point at an ## diagnose.c ## -@@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) +@@ diagnose.c: int create_diagnostics_archive(struct repository *r, struct strvec archiver_args = STRVEC_INIT; char **argv_copy = NULL; int stdout_fd = -1, archiver_fd = -1; @@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnos struct strbuf buf = STRBUF_INIT; int res; struct archive_dir archive_dirs[] = { -@@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) - strbuf_addf(&buf, "Repository root: %s\n", the_repository->worktree); +@@ diagnose.c: int create_diagnostics_archive(struct repository *r, + strbuf_addf(&buf, "Repository root: %s\n", r->worktree); - git_config_get_string("gvfs.cache-server", &cache_server_url); + repo_config_get_string(r, "gvfs.cache-server", &cache_server_url); - strbuf_addf(&buf, "Cache Server: %s\n\n", - cache_server_url ? cache_server_url : "None"); -+ git_config_get_string("gvfs.sharedCache", &shared_cache); ++ repo_config_get_string(r, "gvfs.sharedCache", &shared_cache); + strbuf_addf(&buf, "Cache Server: %s\nLocal Cache: %s\n\n", + cache_server_url ? cache_server_url : "None", + shared_cache ? shared_cache : "None"); get_disk_info(&buf); write_or_die(stdout_fd, buf.buf, buf.len); -@@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) +@@ diagnose.c: int create_diagnostics_archive(struct repository *r, strvec_clear(&archiver_args); strbuf_release(&buf); free(cache_server_url); @@ scalar.c: static int supports_gvfs_protocol(const char *url, char **cache_server + int hash_algo_index = hash_algo_by_name("sha1"); + const struct git_hash_algo *hash_algo = hash_algo_index < 0 ? + the_hash_algo : &hash_algos[hash_algo_index]; -+ git_hash_ctx ctx; ++ struct git_hash_ctx ctx; + unsigned char hash[GIT_MAX_RAWSZ]; + + strbuf_addstr(&downcased, url); @@ scalar.c: void load_builtin_commands(const char *prefix UNUSED, static int cmd_clone(int argc, const char **argv) { const char *branch = NULL; + char *branch_to_free = NULL; int full_clone = 0, single_branch = 0, show_progress = isatty(2); int src = 1, tags = 1; - const char *cache_server_url = NULL; @@ scalar.c: static int cmd_clone(int argc, const char **argv) + trace2_data_intmax("scalar", the_repository, "unattended", + is_unattended()); - if (!branch && !(branch = remote_default_branch(url))) { + if (!branch && !(branch = branch_to_free = remote_default_branch(url))) { res = error(_("failed to get default branch for '%s'"), url); @@ scalar.c: static int cmd_clone(int argc, const char **argv) supports_gvfs_protocol(url, &default_cache_server_url);155: b636dc0 ! 155: 8e41d86 scalar diagnose: include shared cache info
@@ diagnose.c: static int dir_file_stats(struct object_directory *object_dir, void static int count_files(struct strbuf *path) { DIR *dir = opendir(path->buf); -@@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) +@@ diagnose.c: int create_diagnostics_archive(struct repository *r, char **argv_copy = NULL; int stdout_fd = -1, archiver_fd = -1; char *cache_server_url = NULL, *shared_cache = NULL; @@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnos int res; struct archive_dir archive_dirs[] = { { ".git", 0 }, -@@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) +@@ diagnose.c: int create_diagnostics_archive(struct repository *r, } } @@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnos + } + strvec_pushl(&archiver_args, "--prefix=", - oid_to_hex(the_hash_algo->empty_tree), "--", NULL); + oid_to_hex(r->hash_algo->empty_tree), "--", NULL);156: 91ffe1a = 156: 1917e70 scalar: only try GVFS protocol on https:// URLs
157: df4f0a6 = 157: 0e76a11 scalar: verify that we can use a GVFS-enabled repository
158: 1db87ec ! 158: f7b2abb scalar: add the
cache-servercommand@@ Commit message Signed-off-by: Johannes Schindelin <[email protected]> - ## Documentation/scalar.txt ## -@@ Documentation/scalar.txt: scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) + ## Documentation/scalar.adoc ## +@@ Documentation/scalar.adoc: scalar run ( all | config | commit-graph | fetch | loose-objects | pack-files ) scalar reconfigure [ --all | <enlistment> ] scalar diagnose [<enlistment>] scalar delete <enlistment> @@ Documentation/scalar.txt: scalar run ( all | config | commit-graph | fetch | loo DESCRIPTION ----------- -@@ Documentation/scalar.txt: delete <enlistment>:: +@@ Documentation/scalar.adoc: delete <enlistment>:: This subcommand lets you delete an existing Scalar enlistment from your local file system, unregistering the repository.159: 380bb89 = 159: d84ca9b scalar: add a test toggle to skip accessing the vsts/info endpoint
160: 54783df = 160: 58f9f2e scalar: adjust documentation to the microsoft/git fork
161: 22e1322 = 161: 5224e24 scalar: enable untracked cache unconditionally
162: 8116d20 ! 162: 04a0bfa scalar: parse
clone --no-fetch-commits-and-treesfor backwards compatibility@@ scalar.c: static int init_shared_object_cache(const char *url, { + int dummy = 0; const char *branch = NULL; + char *branch_to_free = NULL; int full_clone = 0, single_branch = 0, show_progress = isatty(2); - int src = 1, tags = 1; @@ scalar.c: static int cmd_clone(int argc, const char **argv) OPT_STRING(0, "local-cache-path", &local_cache_root, N_("<path>"),163: 21fa0fd ! 163: 914f1f9 scalar: make GVFS Protocol a forced choice
@@ Commit message Signed-off-by: Derrick Stolee <[email protected]> - ## Documentation/scalar.txt ## -@@ Documentation/scalar.txt: clone), and `~/.scalarCache` on macOS. + ## Documentation/scalar.adoc ## +@@ Documentation/scalar.adoc: clone), and `~/.scalarCache` on macOS. Retrieve missing objects from the specified remote, which is expected to understand the GVFS protocol.164: becf22c ! 164: 4cbdb67 scalar diagnose: accommodate Scalar's Functional Tests
@@ Commit message Signed-off-by: Johannes Schindelin <[email protected]> ## diagnose.c ## -@@ diagnose.c: int create_diagnostics_archive(struct strbuf *zip_path, enum diagnose_mode mode) +@@ diagnose.c: int create_diagnostics_archive(struct repository *r, goto diagnose_cleanup; }165: efd5b8f = 165: 7ab5902 ci: run Scalar's Functional Tests
166: aae1e8f = 166: 3c48ffb scalar: upgrade to newest FSMonitor config setting
167: aa3fed6 = 167: 11bd941 abspath: make strip_last_path_component() global
168: 19134cb = 168: 0ed3dcc scalar: configure maintenance during 'reconfigure'
169: b47a29c = 169: d3ed86f scalar: .scalarCache should live above enlistment
170: 4250d6d = 170: 4e1b1f6 add/rm: allow adding sparse entries when virtual
171: a133894 ! 171: 2da0919 sparse-checkout: add config to disable deleting dirs
@@ Commit message Signed-off-by: Derrick Stolee <[email protected]> - ## Documentation/config/index.txt ## + ## Documentation/config/index.adoc ## @@ +index.deleteSparseDirectories:: + When enabled, the cone mode sparse-checkout feature will delete172: f053ba3 = 172: ea7cae2 diff: ignore sparse paths in diffstat
173: 6e6b8fc = 173: 0a22b84 repo-settings: enable sparse index by default
174: 8fff938 = 174: 214578f diff(sparse-index): verify with partially-sparse
175: d6f8f4c = 175: e8c0e72 stash: expand testing for
git stash -u176: 1df7b6f = 176: b49b34d sequencer: avoid progress when stderr is redirected
177: 528521c = 177: c64b811 sparse: add vfs-specific precautions
178: 8b4a143 = 178: 4f89d60 reset: fix mixed reset when using virtual filesystem
179: 1991d52 = 179: 324ad83 sparse-index: add ensure_full_index_with_reason()
180: 7a919dc = 180: a8449e1 treewide: add reasons for expanding index
181: 74f0e1c = 181: 04ea9d0 treewide: custom reasons for expanding index
182: d89823c ! 182: 021e269 sparse-index: add macro for unaudited expansions
183: 82f663a ! 183: 220f41f Docs: update sparse index plan with logging
@@ Commit message Signed-off-by: Derrick Stolee <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]> - ## Documentation/technical/sparse-index.txt ## -@@ Documentation/technical/sparse-index.txt: Here are some commands that might be useful to update: + ## Documentation/technical/sparse-index.adoc ## +@@ Documentation/technical/sparse-index.adoc: Here are some commands that might be useful to update: * `git am` * `git clean` * `git stash`184: d86e967 = 184: c87d5c0 sparse-index: log failure to clear skip-worktree
185: d4f8e8c = 185: 1255e3f stash: use -f in checkout-index child process
186: 05dca1b = 186: c8fadbc sparse-index: do not copy hashtables during expansion
187: 501634b < -: ------------ t5616: mark tests as bogus with --path-walk
188: 2e31da5 < -: ------------ path-walk: add new 'edge_aggressive' option
-: ------------ > 187: ff6c4d7 path-walk: add new 'edge_aggressive' option
189: 4a7f1dd = 188: b18d1a0 pack-objects: allow --shallow and --path-walk
190: 19094f4 = 189: 7d95375 t5538: add test to confirm deltas in shallow pushes
191: 71e11fb = 190: 8dddd73 sub-process: avoid leaking
cmd192: 78daa0b = 191: 927c26c remote-curl: release filter options before re-setting them
193: 4d43345 = 192: 394a129 transport: release object filter options
194: 9551455 = 193: b7f2e33 mingw: special-case administrators even more
195: be80005 = 194: 3fc340b test-tool path-utils: support debugging "dubious ownership" issues
196: c775ce5 ! 195: 65ba23c push: don't reuse deltas with path walk
197: 134fb83 = 196: 04efb4a release-winget.yml: update command to include ARM installers
198: 63d80ba = 197: a8f8c2d t7900-maintenance.sh: reset config between tests
199: 7c53544 ! 198: df89b84 maintenance: add cache-local-objects maintenance task
@@ Commit message Signed-off-by: Matthew John Cheetham <[email protected]> - ## Documentation/git-maintenance.txt ## -@@ Documentation/git-maintenance.txt: task: + ## Documentation/git-maintenance.adoc ## +@@ Documentation/git-maintenance.adoc: task: * `prefetch`: hourly. * `loose-objects`: daily. * `incremental-repack`: daily. @@ Documentation/git-maintenance.txt: task: -- + `git maintenance register` will also disable foreground maintenance by -@@ Documentation/git-maintenance.txt: pack-refs:: +@@ Documentation/git-maintenance.adoc: pack-refs:: need to iterate across many references. See linkgit:git-pack-refs[1] for more information.200: d0967a3 = 199: bbbe8fa scalar.c: add cache-local-objects task
201: bda3f91 < -: ------------ fixup! update-microsoft-git: create barebones builtin
202: 627a6bf < -: ------------ fixup! release: create initial Windows installer build workflow
203: 842cfa4 < -: ------------ fixup! release: build unsigned Ubuntu .deb package
204: dc81d69 < -: ------------ fixup! release: add Mac OSX installer build
205: a053985 < -: ------------ fixup! Adding winget workflows
206: fec16ed < -: ------------ fixup! build-git-installers: publish gpg public key