Weekly Digest (11 November, 2018 - 18 November, 2018)

[weekly-digest]

Here's the Weekly Digest for git-for-windows/git:

---

ISSUES

Last week 10 issues were created.
Of these, 5 issues have been closed and 5 issues are still open.

OPEN ISSUES

💚 #1934 fscache: make fscache_enable() thread safe, by benpeart
💚 #1931 Alt-F2 no longer opens a new window, by akerfoot
💚 #1930 Spurious Carriage Return breaks diff tool, by stravant
💚 #1927 Windows Sever 2008 R2 graceful handing of pre-receive hooks, by JayD123
💚 #1925 home folder can not easily be modified, by f-steff

CLOSED ISSUES

❤️ #1933 Update create-empty-bundle, by dscho
❤️ #1932 Prepare for v2.20.0-rc0, by dscho
❤️ #1928 This is a bug; please report it at, by william-denault
❤️ #1926 Improve performance of fscache, by benpeart
❤️ #1924 Cannot do git checkout -- . or git reset --hard when affected files' mode changed from 100755 to 100644, by chenzx

NOISY ISSUE

🔈 #1924 Cannot do git checkout -- . or git reset --hard when affected files' mode changed from 100755 to 100644, by chenzx
It received 7 comments.

---

PULL REQUESTS

Last week, 8 pull requests were created, updated or merged.

OPEN PULL REQUEST

Last week, 1 pull request was opened.
💚 #1934 fscache: make fscache_enable() thread safe, by benpeart

UPDATED PULL REQUEST

Last week, 1 pull request was updated.
💛 #1902 [Outreachy] config support for allowEmpty in commit, by tanushree27

MERGED PULL REQUEST

Last week, 6 pull requests were merged.
💜 #1933 Update create-empty-bundle, by dscho
💜 #1932 Prepare for v2.20.0-rc0, by dscho
💜 #1926 Improve performance of fscache, by benpeart
💜 #1914 At the end of the add command, disable and free the fscache, by benpeart
💜 #1910 fscache: add fscache hit statistics, by benpeart
💜 #1900 [Outreachy] Removed ipv6 fallback, by tanushree27

---

COMMITS

Last week there were 175 commits.
🛠️ Merge pull request #1933 from dscho/update-create-empty-bundle Update create-empty-bundle by dscho
🛠️ Merge pull request #1932 from dscho/prepare-for-2.20.0-rc0 Prepare for v2.20.0-rc0 by dscho
🛠️ squash! bundle: refuse to create empty bundle TODO: change authorship to Peff! bundle: dup() output descriptor closer to point-of-use When writing a bundle to a file, the bundle code actually creates "your.bundle.lock" using our lockfile interface. We feed that output descriptor to a child git-pack-objects via run-command, which has the quirk that it closes the output descriptor in the parent. To avoid confusing the lockfile code (which still thinks the descriptor is valid), we dup() it, and operate on the duplicate. However, this has a confusing side effect: after the dup() but before we call pack-objects, we have two descriptors open to the lockfile. If we call die() during that time, the lockfile code will try to clean up the partially-written file. It knows to close() the file before unlinking, since on some platforms (i.e., Windows) the open file would block the deletion. But it doesn't know about the duplicate descriptor. On Windows, triggering an error at the right part of the code will result in the cleanup failing and the lockfile being left in the filesystem. We can solve this by moving the dup() much closer to start_command(), shrinking the window in which we have the second descriptor open. It's easy to place this in such a way that no die() is possible. We could still die due to a signal in the exact wrong moment, but we already tolerate races there (e.g., a signal could come before we manage to put the file on the cleanup list in the first place). As a bonus, this shields create_bundle() itself from the duplicate-fd trick, and we can simplify its error handling (note that the lock rollback now happens unconditionally, but that's OK; it's a noop if we didn't open the lock in the first place). The included test uses an empty bundle to cause a failure at the right spot in the code, because that's easy to trigger (the other likely errors are write() problems like ENOSPC). Note that it would already pass on non-Windows systems (because they are happy to unlink an already-open file). Based-on-a-patch-by: Gaël Lhez [email protected] Signed-off-by: Jeff King [email protected] Signed-off-by: Johannes Schindelin [email protected] by peff
🛠️ fixup! bundle: refuse to create empty bundle This reverts the patch we carried for a long time, as it did not survive contact with the Git mailing list. In preparation for applying Jeff King's better version, let's revert our version. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge remote-tracking branch 'benpeart/fscache-per-thread-gfw' This brings substantial wins in performance because the FSCache is now per-thread, being merged to the primary thread only at the end, so we do not have to lock (except while merging). Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fscache: teach fscache to use mempool Now that the fscache is single threaded, take advantage of the mem_pool as the allocator to significantly reduce the cost of allocations and frees. With the reduced cost of free, in future patches, we can start freeing the fscache at the end of commands instead of just leaking it. Signed-off-by: Ben Peart [email protected] by benpeart
🛠️ fscache: update fscache to be thread specific instead of global The threading model for fscache has been to have a single, global cache. This puts requirements on it to be thread safe so that callers like preload-index can call it from multiple threads. This was implemented with a single mutex and completion events which introduces contention between the calling threads. Simplify the threading model by making fscache thread specific. This allows us to remove the global mutex and synchronization events entirely and instead associate a fscache with every thread that requests one. This works well with the current multi-threading which divides the cache entries into blocks with a separate thread processing each block. At the end of each worker thread, if there is a fscache on the primary thread, merge the cached results from the worker into the primary thread cache. This enables us to reuse the cache later especially when scanning for untracked files. In testing, this reduced the time spent in preload_index() by about 25% and also reduced the CPU utilization significantly. On a repo with ~200K files, it reduced overall status times by ~12%. Signed-off-by: Ben Peart [email protected] by benpeart
🛠️ fscache: fscache takes an initial size Update enable_fscache() to take an optional initial size parameter which is used to initialize the hashmap so that it can avoid having to rehash as additional entries are added. Add a separate disable_fscache() macro to make the code clearer and easier to read. Signed-off-by: Ben Peart [email protected] by benpeart
🛠️ Merge pull request #1910 from benpeart/fscache_statistics-gfw fscache: add fscache hit statistics by dscho
🛠️ Merge pull request #1914 from benpeart/free-fscache-after-add-gfw At the end of the add command, disable and free the fscache by dscho
🛠️ fixup! Add a build definition for Azure DevOps Let's use the "Hosted" pool, it seems to be under less load, i.e. it will make our builds faster. While at it, also disable chain linting and bin-wrappers in the Windows phase, to save even more time. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ tests: optionally skip bin-wrappers/ This speeds up the tests by a bit on Windows, where running Unix shell scripts (and spawning processes) is not exactly a cheap operation. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ t0061: fix with --with-dashes and RUNTIME_PREFIX When building Git with RUNTIME_PREFIX and starting a test helper from t/helper/, it fails to detect the system prefix correctly. This is the reason that the warning RUNTIME_PREFIX requested, but prefix computation failed. [...] to be printed. In t0061, we did not expect that to happen, and it actually did not happen in the normal case, because bin-wrappers/test-tool specifically sets GIT_TEXTDOMAINDIR (and as a consequence, nothing in test-tool wants to know about the runtime prefix). However, with --with-dashes, bin-wrappers/test-tool is no longer called, but t/helper/test-tool is called directly. So let's just ignore the RUNTIME_PREFIX warning. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ squash! tests: explicitly use git.exe on Windows For the same reason, we need to handle test-tool.exe the same way. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ tests: add t/helper/ to the PATH with --with-dashes We really need to be able to find the test helpers... Really. This change was forgotten when we moved the test helpers into t/helper/ Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fixup! stash/rebase: default to the non-builtin versions In Git for Windows v2.20.0, we will want to default to the built-in versions. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge branch 'vcxproj' into prepare-for-2.20.0-rc0 These patches will supersede the corresponding patches that are already in the branch thicket, during the next merging-rebase. That merging-rebase is scheduled for v2.20.0-rc0, which is scheduled later tonight. However, the plan is to hold off with pushing this to master until the merging-rebase to v2.20.0-rc1. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge branch 'kblees/kb/symlinks' by dscho
🛠️ Merge branch 'msys2' by dscho
🛠️ Merge branch 'long-paths' by dscho
🛠️ Merge branch 'fscache' by dscho
🛠️ Merge branch 'visual-studio' Signed-off-by: Johannes Schindelin [email protected] by jeffhostetler
🛠️ Merge branch 'msvc' Signed-off-by: Johannes Schindelin [email protected] by jeffhostetler
🛠️ mingw: try to create symlinks without elevated permissions With Windows 10 Build 14972 in Developer Mode, a new flag is supported by CreateSymbolicLink() to create symbolic links even when running outside of an elevated session (which was previously required). This new flag is called SYMBOLIC_LINK_FLAG_ALLOW_UNPRIVILEGED_CREATE and has the numeric value 0x02. Previous Windows 10 versions will not understand that flag and return an ERROR_INVALID_PARAMETER, therefore we have to be careful to try passing that flag only when the build number indicates that it is supported. For more information about the new flag, see this blog post: https://blogs.windows.com/buildingapps/2016/12/02/symlinks-windows-10/ This patch is loosely based on the patch submitted by Samuel D. Leslie as https://github.com/git-for-windows/git/pull/1184. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Win32: symlink: add support for symlinks to directories Symlinks on Windows have a flag that indicates whether the target is a file or a directory. Symlinks of wrong type simply don't work. This even affects core Win32 APIs (e.g. DeleteFile() refuses to delete directory symlinks). However, CreateFile() with FILE_FLAG_BACKUP_SEMANTICS doesn't seem to care. Check the target type by first creating a tentative file symlink, opening it, and checking the type of the resulting handle. If it is a directory, recreate the symlink with the directory flag set. It is possible to create symlinks before the target exists (or in case of symlinks to symlinks: before the target type is known). If this happens, create a tentative file symlink and postpone the directory decision: keep a list of phantom symlinks to be processed whenever a new directory is created in mingw_mkdir(). Limitations: This algorithm may fail if a link target changes from file to directory or vice versa, or if the target directory is created in another process. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Win32: implement basic symlink() functionality (file symlinks only) Implement symlink() that always creates file symlinks. Fails with ENOSYS if symlinks are disabled or unsupported. Note: CreateSymbolicLinkW() was introduced with symlink support in Windows Vista. For compatibility with Windows XP, we need to load it dynamically and fail gracefully if it isnt's available. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: implement readlink() Implement readlink() by reading NTFS reparse points. Works for symlinks and directory junctions. If symlinks are disabled, fail with ENOSYS. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: mingw_chdir: change to symlink-resolved directory If symlinks are enabled, resolve all symlinks when changing directories, as required by POSIX. Note: Git's real_path() function bases its link resolution algorithm on this property of chdir(). Unfortunately, the current directory on Windows is limited to only MAX_PATH (260) characters. Therefore using symlinks and long paths in combination may be problematic. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ Win32: mingw_rename: support renaming symlinks MSVCRT's _wrename() cannot rename symlinks over existing files: it returns success without doing anything. Newer MSVCR*.dll versions probably do not have this problem: according to CRT sources, they just call MoveFileEx() with the MOVEFILE_COPY_ALLOWED flag. Get rid of _wrename() and call MoveFileEx() with proper error handling. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: mingw_unlink: support symlinks to directories _wunlink() / DeleteFileW() refuses to delete symlinks to directories. If _wunlink() fails with ERROR_ACCESS_DENIED, try _wrmdir() as well. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: add symlink-specific error codes Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: change default of 'core.symlinks' to false Symlinks on Windows don't work the same way as on Unix systems. E.g. there are different types of symlinks for directories and files, creating symlinks requires administrative privileges etc. By default, disable symlink support on Windows. I.e. users explicitly have to enable it with 'git config [--system|--global] core.symlinks true'. The test suite ignores system / global config files. Allow testing with symlink support by checking if native symlinks are enabled in MSys2 (via 'MSYS=winsymlinks:nativestrict'). Reminder: This would need to be changed if / when we find a way to run the test suite in a non-MSys-based shell (e.g. dash). Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: factor out retry logic The retry pattern is duplicated in three places. It also seems to be too hard to use: mingw_unlink() and mingw_rmdir() duplicate the code to retry, and both of them do so incompletely. They also do not restore errno if the user answers 'no'. Introduce a retry_ask_yes_no() helper function that handles retry with small delay, asking the user, and restoring errno. mingw_unlink: include _wchmod in the retry loop (which may fail if the file is locked exclusively). mingw_rmdir: include special error handling in the retry loop. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: lstat(): return adequate stat.st_size for symlinks Git typically doesn't trust the stat.st_size member of symlinks (e.g. see strbuf_readlink()). However, some functions take shortcuts if st_size is 0 (e.g. diff_populate_filespec()). In mingw_lstat() and fscache_lstat(), make sure to return an adequate size. The extra overhead of opening and reading the reparse point to calculate the exact size is not necessary, as git doesn't rely on the value anyway. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ Win32: teach fscache and dirent about symlinks Move S_IFLNK detection to file_attr_to_st_mode() and reuse it in fscache. Implement DT_LNK detection in dirent.c and the fscache readdir version. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ Win32: let mingw_lstat() error early upon problems with reparse points When obtaining lstat information for reparse points, we need to call FindFirstFile() in addition to GetFileInformationEx() to obtain the type of the reparse point (symlink, mount point etc.). However, currently there is no error handling whatsoever if FindFirstFile() fails. Call FindFirstFile() before modifying the stat *buf output parameter and error out if the call fails. Note: The FindFirstFile() return value includes all the data that we get from GetFileAttributesEx(), so we could replace GetFileAttributesEx() with FindFirstFile(). We don't do that because GetFileAttributesEx() is about twice as fast for single files. I.e. we only pay the extra cost of calling FindFirstFile() in the rare case that we encounter a reparse point. Note: The indentation of the remaining reparse point code will be fixed in the next patch. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: remove separate do_lstat() function With the new mingw_stat() implementation, do_lstat() is only called from mingw_lstat() (with follow == 0). Remove the extra function and the old mingw_stat()-specific (follow == 1) logic. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: implement stat() with symlink support With respect to symlinks, the current stat() implementation is almost the same as lstat(): except for the file type (st_mode & S_IFMT), it returns information about the link rather than the target. Implement stat by opening the file with as little permissions as possible and calling GetFileInformationByHandle on it. This way, all link resoltion is handled by the Windows file system layer. If symlinks are disabled, use lstat() as before, but fail with ELOOP if a symlink would have to be resolved. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ Merge branch 'maybe-drop' These patches are probably no longer necessary. Make sure before dropping them, though. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Win32: don't call GetFileAttributes twice in mingw_lstat() GetFileAttributes cannot handle paths with trailing dir separator. The current [l]stat implementation calls GetFileAttributes twice if the path has trailing slashes (first with the original path passed to [l]stat, and and a second time with a path copy with trailing '/' removed). With Unicode conversion, we get the length of the path for free and also have a (wide char) buffer that can be modified. Remove trailing directory separators before calling the Win32 API. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ t9001: work around hard-to-debug hangs Just like the workaround we added for t9116, t9001.83 hangs sometimes -- but not always! -- when being run in the Git for Windows SDK. The issue seems to be related to redirection via a pipe, but it is really hard to diagnose, what with git.exe (a non-MSYS2 program) calling a Perl script (which is executed by an MSYS2 Perl), piping into another MSYS2 program. As hunting time is scarce these days, simply work around this for now and leave the real diagnosis and resolution for later. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ lockfile.c: use is_dir_sep() instead of hardcoded '/' checks Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ strbuf_readlink: support link targets that exceed PATH_MAX strbuf_readlink() refuses to read link targets that exceed PATH_MAX (even if a sufficient size was specified by the caller). As some platforms support longer paths, remove this restriction (similar to strbuf_getcwd()). Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ strbuf_readlink: don't call readlink twice if hint is the exact link size strbuf_readlink() calls readlink() twice if the hint argument specifies the exact size of the link target (e.g. by passing stat.st_size as returned by lstat()). This is necessary because 'readlink(..., hint) == hint' could mean that the buffer was too small. Use hint + 1 as buffer size to prevent this. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ diff: munmap() file contents before running external diff When running an external diff from, say, a diff tool, it is safe to assume that we want to write the files in question. On Windows, that means that there cannot be any other process holding an open handle to said files. So let's make sure that git diff itself is not holding any open handle to the files in question. This fixes https://github.com/git-for-windows/git/issues/1315 Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ t9116: work around hard-to-debug hangs As of a couple of weeks ago, t9116 hangs sometimes -- but not always! -- when being run in the Git for Windows SDK. The issue seems to be related to redirection via a pipe, but it is really hard to diagnose, what with git.exe (a non-MSYS2 program) calling a Perl script (which is executed by an MSYS2 Perl), piping into another MSYS2 program. As hunting time is scarce these days, simply work around this for now and leave the real diagnosis and resolution for later. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ mingw: make is_hidden tests in t0001/t5611 more robust We should not actually expect the first attrib.exe in the PATH to be the one we are looking for. Or that it is in the PATH, for that matter. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Skip t9020 with MSys2 POSIX-to-Windows path mangling would make it fail. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ mingw: ensure valid CTYPE A change between versions 2.4.1 and 2.6.0 of the MSYS2 runtime modified how Cygwin's runtime (and hence Git for Windows' MSYS2 runtime derivative) handles locales: d16a56306d (Consolidate wctomb/mbtowc calls for POSIX-1.2008, 2016-07-20). An unintended side-effect is that "cold-calling" into the POSIX emulation will start with a locale based on the current code page, something that Git for Windows is very ill-prepared for, as it expects to be able to pass a command-line containing non-ASCII characters to the shell without having those characters munged. One symptom of this behavior: when git clone or git fetch shell out to call git-upload-pack with a path that contains non-ASCII characters, the shell tried to interpret the entire command-line (including command-line parameters) as executable path, which obviously must fail. This fixes https://github.com/git-for-windows/git/issues/1036 Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Tests: optionally skip redirecting stdin/stdout/stderr There is a really useful debugging technique developed by Sverre Rabbelier that inserts "bash &&" somewhere in the test scripts, letting the developer interact at given points with the current state. Another debugging technique, used a lot by this here coder, is to run certain executables via gdb by guarding a "gdb -args" call in bin-wrappers/git. Both techniques were disabled by 781f76b1(test-lib: redirect stdin of tests). Let's reinstate the ability to run an interactive shell by making the redirection optional: setting the TEST_NO_REDIRECT environment variable will skip the redirection. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Unbreak interactive GPG prompt upon signing With the recent update in efee955 (gpg-interface: check gpg signature creation status, 2016-06-17), we ask GPG to send all status updates to stderr, and then catch the stderr in an strbuf. But GPG might fail, and send error messages to stderr. And we simply do not show them to the user. Even worse: this swallows any interactive prompt for a passphrase. And detaches stderr from the tty so that the passphrase cannot be read. So while the first problem could be fixed (by printing the captured stderr upon error), the second problem cannot be easily fixed, and presents a major regression. So let's just revert commit efee9553a4f97b2ecd8f49be19606dd4cf7d9c28. This fixes https://github.com/git-for-windows/git/issues/871 Cc: Michael J Gruber [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ MinGW: Use MakeMaker to build the Perl libraries This way the libraries get properly installed into the "site_perl" directory and we just have to move them out of the "mingw" directory. Signed-off-by: Sebastian Schuberth [email protected] by sschuberth
🛠️ winansi: simplify loading the GetCurrentConsoleFontEx() function We introduced helper macros to simplify loading functions dynamically. Might just as well use them. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ compat/terminal.c: only use the Windows console if bash 'read -r' fails Accessing the Windows console through the special CONIN$ / CONOUT$ devices doesn't work properly for non-ASCII usernames an passwords. It also doesn't work for terminal emulators that hide the native console window (such as mintty), and 'TERM=xterm*' is not necessarily a reliable indicator for such terminals. The new shell_prompt() function, on the other hand, works fine for both MSys1 and MSys2, in native console windows as well as mintty, and properly supports Unicode. It just needs bash on the path (for 'read -s', which is bash-specific). On Windows, try to use the shell to read from the terminal. If that fails with ENOENT (i.e. bash was not found), use CONIN/OUT as fallback. Note: To test this, create a UTF-8 credential file with non-ASCII chars, e.g. in git-bash: 'echo url=http://täst.com > cred.txt'. Then in git-cmd, 'git credential fill <cred.txt' works (shell version), while calling git without the git-wrapper (i.e. 'mingw64\bin\git credential fill <cred.txt') mangles non-ASCII chars in both console output and input. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ mingw: Support git_terminal_prompt with more terminals The git_terminal_prompt() function expects the terminal window to be attached to a Win32 Console. However, this is not the case with terminal windows other than cmd.exe's, e.g. with MSys2's own mintty. Non-cmd terminals such as mintty still have to have a Win32 Console to be proper console programs, but have to hide the Win32 Console to be able to provide more flexibility (such as being resizeable not only vertically but also horizontally). By writing to that Win32 Console, git_terminal_prompt() manages only to send the prompt to nowhere and to wait for input from a Console to which the user has no access. This commit introduces a function specifically to support mintty -- or other terminals that are compatible with MSys2's /dev/tty emulation. We use the TERM environment variable as an indicator for that: if the value starts with "xterm" (such as mintty's "xterm_256color"), we prefer to let xterm_prompt() handle the user interaction. The most prominent user of git_terminal_prompt() is certainly git-remote-https.exe. It is an interesting use case because both stdin and stdout are redirected when Git calls said executable, yet it still wants to access the terminal. When running inside a mintty, the terminal is not accessible to the git-remote-https.exe program, though, because it is a MinGW program and the mintty terminal is not backed by a Win32 console. To solve that problem, we simply call out to the shell -- which is an MSys2 program and can therefore access /dev/tty. Helped-by: nalla [email protected] Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ mingw: explicitly fflush stdout For performance reasons stdout is not unbuffered by default. That leads to problems if after printing to stdout a read on stdin is performed. For that reason interactive commands like git clean -i do not function properly anymore if the stdout is not flushed by fflush(stdout) before trying to read from stdin. In the case of git clean -i all reads on stdin were preceded by a fflush(stdout) call. Signed-off-by: nalla [email protected] by nalla
🛠️ mingw: initialize HOME on startup HOME initialization was historically duplicated in many different places, including /etc/profile, launch scripts such as git-bash.vbs and gitk.cmd, and (although slightly broken) in the git-wrapper. Even unrelated projects such as GitExtensions and TortoiseGit need to implement the same logic to be able to call git directly. Initialize HOME in git's own startup code so that we can eventually retire all the duplicate initialization code. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ gettext: always use UTF-8 on native Windows Git on native Windows exclusively uses UTF-8 for console output (both with mintty and native console windows). Gettext uses setlocale() to determine the output encoding for translated text, however, MSVCRT's setlocale() doesn't support UTF-8. As a result, translated text is encoded in system encoding (GetAPC()), and non-ASCII chars are mangled in console output. Use gettext's bind_textdomain_codeset() to force the encoding to UTF-8 on native Windows. In this developers' setup, HAVE_LIBCHARSET_H is apparently defined, but we really want to override the locale_charset() here. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Avoid illegal filenames when building Documentation on NTFS A '+' is not a valid part of a filename with Windows file systems (it is reserved because the '+' operator meant file concatenation back in the DOS days). Let's just not use it. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ mingw: enable stack smashing protector As suggested privately to Brendan Forster by some unnamed person (suggestion for the future: use the public mailing list, or even the public GitHub issue tracker, that is a much better place to offer such suggestions), we should make use of gcc's stack smashing protector that helps detect stack buffer overruns early. Rather than using -fstack-protector, we use -fstack-protector-strong because it strikes a better balance between how much code is affected and the performance impact. In a local test (time git log --grep=is -p), best of 5 timings went from 23.009s to 22.997s (i.e. the performance impact was well lost in the noise). This fixes https://github.com/git-for-windows/git/issues/501 Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ mingw: Embed a manifest to trick UAC into Doing The Right Thing On Windows >= Vista, not having an application manifest with a requestedExecutionLevel can cause several kinds of confusing behavior. The first and more obvious behavior is "Installer Detection", where Windows sometimes decides (by looking at things like the file name and even sequences of bytes within the executable) that an executable is an installer and should run elevated (causing the well-known popup dialog to appear). In Git's context, subcommands such as "git patch-id" or "git update-index" fall prey to this behavior. The second and more confusing behavior is "File Virtualization". It means that when files are written without having write permission, it does not fail (as expected), but they are instead redirected to somewhere else. When the files are read, the original contents are returned, though, not the ones that were just written somewhere else. Even more confusing, not all write accesses are redirected; Trying to write to write-protected .exe files, for example, will fail instead of redirecting. In addition to being unwanted behavior, File Virtualization causes dramatic slowdowns in Git (see for instance http://code.google.com/p/msysgit/issues/detail?id=320). There are two ways to prevent those two behaviors: Either you embed an application manifest within all your executables, or you add an external manifest (a file with the same name followed by .manifest) to all your executables. Since Git's builtins are hardlinked (or copied), it is simpler and more robust to embed a manifest. A recent enough MSVC compiler should already embed a working internal manifest, but for MinGW you have to do so by hand. Very lightly tested on Wine, where like on Windows XP it should not make any difference. References: - New UAC Technologies for Windows Vista http://msdn.microsoft.com/en-us/library/bb756960.aspx - Create and Embed an Application Manifest (UAC) http://msdn.microsoft.com/en-us/library/bb756929.aspx [js: simplified the embedding dramatically by reusing Git for Windows' existing Windows resource file, removed the optional (and dubious) processorArchitecture attribute of the manifest's assemblyIdentity section.] Signed-off-by: Cesar Eduardo Barros [email protected] Signed-off-by: Johannes Schindelin [email protected] by cesarb
🛠️ Build Python stuff with MSys2 Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Help debugging with MSys2 by optionally executing bash with strace MSys2's strace facility is very useful for debugging... With this patch, the bash will be executed through strace if the environment variable GIT_STRACE_COMMANDS is set, which comes in real handy when investigating issues in the test suite. Also support passing a path to a log file via GIT_STRACE_COMMANDS to force Git to call strace.exe with the -o <path> argument, i.e. to log into a file rather than print the log directly. That comes in handy when the output would otherwise misinterpreted by a calling process as part of Git's output. Note: the values "1", "yes" or "true" are not specifying paths, but tell Git to let strace.exe log directly to the console. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge branch 'unc-path-w-backslashes' Merging this topic branch because it would otherwise conflict with the next commit. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Makefile: Set htmldir to match the default HTML docs location under MSYS Signed-off-by: Sebastian Schuberth [email protected] by sschuberth
🛠️ Win32: fix 'lstat("dir/")' with long paths Use a suffciently large buffer to strip the trailing slash. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: support long paths Windows paths are typically limited to MAX_PATH = 260 characters, even though the underlying NTFS file system supports paths up to 32,767 chars. This limitation is also evident in Windows Explorer, cmd.exe and many other applications (including IDEs). Particularly annoying is that most Windows APIs return bogus error codes if a relative path only barely exceeds MAX_PATH in conjunction with the current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG. Many Windows wide char APIs support longer than MAX_PATH paths through the file namespace prefix ('\?' or '\?\UNC') followed by an absolute path. Notable exceptions include functions dealing with executables and the current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...). Introduce a handle_long_path function to check the length of a specified path properly (and fail with ENAMETOOLONG), and to optionally expand long paths using the '\?' file namespace prefix. Short paths will not be modified, so we don't need to worry about device names (NUL, CON, AUX). Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be limited to MAX_PATH (at least not on Win7), so we can use it to do the heavy lifting of the conversion (translate '/' to '', eliminate '.' and '..', and make an absolute path). Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH limit. Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs that support long paths. While improved error checking is always active, long paths support must be explicitly enabled via 'core.longpaths' option. This is to prevent end users to shoot themselves in the foot by checking out files that Windows Explorer, cmd/bash or their favorite IDE cannot handle. Test suite: Test the case is when the full pathname length of a dir is close to 260 (MAX_PATH). Bug report and an original reproducer by Andrey Rogozhnikov: https://github.com/msysgit/git/pull/122#issuecomment-43604199 Note that the test cannot rely on the presence of short names, as they are not enabled by default except on the system drive. [jes: adjusted test number to avoid conflicts, reinstated && chain, adjusted test to work without short names] Thanks-to: Martin W. Kirst [email protected] Thanks-to: Doug Kelly [email protected] Signed-off-by: Karsten Blees [email protected] Original-test-by: Andrey Rogozhnikov [email protected] Signed-off-by: Stepan Kasal [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ Win32: support long paths Windows paths are typically limited to MAX_PATH = 260 characters, even though the underlying NTFS file system supports paths up to 32,767 chars. This limitation is also evident in Windows Explorer, cmd.exe and many other applications (including IDEs). Particularly annoying is that most Windows APIs return bogus error codes if a relative path only barely exceeds MAX_PATH in conjunction with the current directory, e.g. ERROR_PATH_NOT_FOUND / ENOENT instead of the infinitely more helpful ERROR_FILENAME_EXCED_RANGE / ENAMETOOLONG. Many Windows wide char APIs support longer than MAX_PATH paths through the file namespace prefix ('\?' or '\?\UNC') followed by an absolute path. Notable exceptions include functions dealing with executables and the current directory (CreateProcess, LoadLibrary, Get/SetCurrentDirectory) as well as the entire shell API (ShellExecute, SHGetSpecialFolderPath...). Introduce a handle_long_path function to check the length of a specified path properly (and fail with ENAMETOOLONG), and to optionally expand long paths using the '\?' file namespace prefix. Short paths will not be modified, so we don't need to worry about device names (NUL, CON, AUX). Contrary to MSDN docs, the GetFullPathNameW function doesn't seem to be limited to MAX_PATH (at least not on Win7), so we can use it to do the heavy lifting of the conversion (translate '/' to '', eliminate '.' and '..', and make an absolute path). Add long path error checking to xutftowcs_path for APIs with hard MAX_PATH limit. Add a new MAX_LONG_PATH constant and xutftowcs_long_path function for APIs that support long paths. While improved error checking is always active, long paths support must be explicitly enabled via 'core.longpaths' option. This is to prevent end users to shoot themselves in the foot by checking out files that Windows Explorer, cmd/bash or their favorite IDE cannot handle. Test suite: Test the case is when the full pathname length of a dir is close to 260 (MAX_PATH). Bug report and an original reproducer by Andrey Rogozhnikov: https://github.com/msysgit/git/pull/122#issuecomment-43604199 [jes: adjusted test number to avoid conflicts] Thanks-to: Martin W. Kirst [email protected] Thanks-to: Doug Kelly [email protected] Signed-off-by: Karsten Blees [email protected] Original-test-by: Andrey Rogozhnikov [email protected] Signed-off-by: Stepan Kasal [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Add a test demonstrating a problem with long submodule paths [jes: adjusted test number to avoid conflicts, fixed non-portable use of the 'export' statement, fixed broken && chain] Signed-off-by: Doug Kelly [email protected] Signed-off-by: Johannes Schindelin [email protected] by vangdfang
🛠️ Merge branch 'drive-prefix' This topic branch allows us to specify absolute paths without the drive prefix e.g. when cloning. Example: C:\Users\me> git clone https://github.com/git/git \upstream-git This will clone into a new directory C:\upstream-git, in line with how Windows interprets absolute paths. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge branch 'mingw-CreateHardLink' Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ fscache: load directories only once If multiple threads access a directory that is not yet in the cache, the directory will be loaded by each thread. Only one of the results is added to the cache, all others are leaked. This wastes performance and memory. On cache miss, add a future object to the cache to indicate that the directory is currently being loaded. Subsequent threads register themselves with the future object and wait. When the first thread has loaded the directory, it replaces the future object with the result and notifies waiting threads. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: add a cache below mingw's lstat and dirent implementations Checking the work tree status is quite slow on Windows, due to slow lstat emulation (git calls lstat once for each file in the index). Windows operating system APIs seem to be much better at scanning the status of entire directories than checking single files. Add an lstat implementation that uses a cache for lstat data. Cache misses read the entire parent directory and add it to the cache. Subsequent lstat calls for the same directory are served directly from the cache. Also implement opendir / readdir / closedir so that they create and use directory listings in the cache. The cache doesn't track file system changes and doesn't plug into any modifying file APIs, so it has to be explicitly enabled for git functions that don't modify the working copy. Note: in an earlier version of this patch, the cache was always active and tracked file system changes via ReadDirectoryChangesW. However, this was much more complex and had negative impact on the performance of modifying git commands such as 'git checkout'. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ add infrastructure for read-only file system level caches Add a macro to mark code sections that only read from the file system, along with a config option and documentation. This facilitates implementation of relatively simple file system level caches without the need to synchronize with the file system. Enable read-only sections for 'git status' and preload_index. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: make the lstat implementation pluggable Emulating the POSIX lstat API on Windows via GetFileAttributes[Ex] is quite slow. Windows operating system APIs seem to be much better at scanning the status of entire directories than checking single files. A caching implementation may improve performance by bulk-reading entire directories or reusing data obtained via opendir / readdir. Make the lstat implementation pluggable so that it can be switched at runtime, e.g. based on a config option. Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by kblees
🛠️ fixup! builtin rebase: call git am directly by dscho
🛠️ fscache: add fscache hit statistics Add cache hit/miss statistics to the fscache for lstat() and opendir(). The statistics are printed out when the cache is disabled and cleared and only if GIT_TRACE_FSCACHE is set. Signed-off-by: Ben Peart [email protected] by benpeart
🛠️ mem_pool: add GIT_TRACE_MEMPOOL support Add tracing around initializing and discarding mempools. In discard report on the amount of memory unused in the current block to help tune setting the initial_size. Signed-off-by: Ben Peart [email protected] by benpeart
🛠️ Merge branch 'js/builtin-rebase-perf-fix' by dscho
🛠️ Merge branch 'js/rebase-autostash-detach-fix' by dscho
🛠️ Merge branch 'js/rebase-autostash-fix' by dscho
🛠️ Merge branch 'jc/rebase-in-c-5-test-typofix' by dscho
🛠️ built-in rebase: reinstate checkout -q behavior where appropriate When we converted a git checkout -q $onto^0 call to use reset_head(), we inadvertently incurred a change from a twoway_merge to a oneway_merge, as if we wanted a git reset --hard instead. This has performance ramifications under certain, though, as the oneway_merge needs to lstat() every single index entry whereas twoway_merge does not. So let's go back to the old behavior. Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase: prepare reset_head() for more flags Currently, we only accept the flag indicating whether the HEAD should be detached not. In the next commit, we want to introduce another flag: to toggle between emulating reset --hard vs checkout -q. Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase: consolidate clean-up code before leaving reset_head() The same clean-up code is repeated quite a few times; Let's DRY up the code some. Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ built-in rebase --autostash: leave the current branch alone if possible When we converted a git reset --hard call in the original Unix shell script to built-in code, we asked to reset the worktree and the index and explicitly not to detach the HEAD. By mistake, though, we still did. Let's fix this. Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase --autostash: fix issue with dirty submodules Since we cannot stash dirty submodules, there is no use in requiring them to be clean (or stash them when they are not). This brings the built-in rebase in line with the previous, scripted version, which also did not care about dirty submodules (but it was admittedly not very easy to figure that out). This fixes https://github.com/git-for-windows/git/issues/1820 Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ built-in rebase: demonstrate regression with --autostash An unnamed colleague of Ævar Arnfjörð Bjarmason reported a breakage where a pull --rebase (which did not really need to do anything but stash, see that nothing was changed, and apply the stash again) also detached the HEAD. This patch adds a minimal reproducer for this regression. Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase --autostash: demonstrate a problem with dirty submodules It has been reported that dirty submodules cause problems with the built-in rebase when it is asked to autostash. The symptom is: fatal: Unexpected stash response: '' This patch adds a regression test that demonstrates that bug. Original report: https://github.com/git-for-windows/git/issues/1820 Signed-off-by: Johannes Schindelin [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase: fix typoes in error messages The separator between words in a multi-word option name is a dash, not an underscore. Inspired by a matching change by Ralf Thielow for the scripted version of "git rebase". Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by gitster
🛠️ rebase (autostash): use an explicit OID to apply the stash When git stash apply <argument> sees an argument that consists only of digits, it tries to be smart and interpret it as stash@{<number>}. Unfortunately, an all-digit hash (which is unlikely but still possible) is therefore misinterpreted as stash@{<n>} reflog. To prevent that from happening, let's append ^0 after the stash hash, to make sure that it is interpreted as an OID rather than as a number. Signed-off-by: Johannes Schindelin [email protected] Reviewed-by: SZEDER Gábor [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase (autostash): store the full OID in /autostash It was reported by Gábor Szeder and analyzed by Alban Gruin that the built-in rebase stores only abbreviated stash hashes in the autostash file. This is problematic e.g. in t5520-pull.sh, where the abbreviated hash is so short that it sometimes consists only of digits, which are subsequently mistaken ("DWIMmed") for numbers by git stash apply. Let's align the behavior of the built-in rebase with the scripted rebase and store the full stash hash instead. That makes it a lot less likely that it consists only of digits. Signed-off-by: Johannes Schindelin [email protected] Reviewed-by: SZEDER Gábor [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase (autostash): avoid duplicate call to state_dir_path() We already called that function at this point, and stored the result in the path variable. We might just as well use it ;-) Signed-off-by: Johannes Schindelin [email protected] Reviewed-by: SZEDER Gábor [email protected] Signed-off-by: Junio C Hamano [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge branch 'ps/stash-in-c-v10-pre1' by dscho
🛠️ stash: optionally use the scripted version again We recently converted the git stash command from Unix shell scripts to builtins. Just like we have rebase.useBuiltin to fall back to the scripted rebase, to give end users a way out when they discover a bug in the builtin command, this commit adds support for stash.useBuiltin. This is necessary because Git for Windows wants to ship the builtin stash earlier than core Git: Git for Windows v2.19.0 will come with the option of a drastically faster (if a lot less battle-tested) git stash. As the file name git-stash is already in use, let's rename the scripted backend to git-legacy-stash. To make the test suite pass with stash.useBuiltin=false, this commit also backports rudimentary support for -q (but only just enough to appease the test suite), and adds a super-ugly hack to force exit code 129 for git stash -h. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Add back the original, scripted git stash This simply copies the version as of v2.19.0-rc0 verbatim. As of now, it is not hooked up. The next commit will change the builtin stash to hand off to the scripted git stash when stash.useBuiltin=false. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ stash: replace all write-tree child processes with API calls This commit replaces spawning git write-tree with API calls. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: optimize get_untracked_files() and check_changes() This commits introduces a optimization by avoiding calling the same functions again. For example, git stash push -u would call at some points the following functions: * check_changes() (inside do_push_stash()) * do_create_stash(), which calls: check_changes() and get_untracked_files() Note that check_changes() also calls get_untracked_files(). So, check_changes() is called 2 times and get_untracked_files() 3 times. The old function check_changes() now consists of two functions: get_untracked_files() and check_changes_tracked_files(). These are the call chains for push and create: * push_stash() -> do_push_stash() -> do_create_stash() * create_stash() -> do_create_stash() To prevent calling the same functions over and over again, check_changes() inside do_create_stash() is now placed in the caller functions (create_stash() and do_push_stash()). This way check_changes() and get_untracked files() are called only one time. https://public-inbox.org/git/[email protected]/ Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert stash--helper.c into stash.c The old shell script git-stash.sh was removed and replaced entirely by builtin/stash.c. In order to do that, create and push were adapted to work without stash.sh. For example, before this commit, git stash create called git stash--helper create --message "$*". If it called git stash--helper create "$@", then some of these changes wouldn't have been necessary. This commit also removes the word helper since now stash is called directly and not by a shell script. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert save to builtin Add stash save to the helper and delete functions which are no longer needed (show_help(), save_stash(), push_stash(), create_stash(), clear_stash(), untracked_files() and no_changes()). Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: make push -q quiet There is a change in behaviour with this commit. When there was no initial commit, the shell version of stash would still display a message. This commit makes push to not display any message if --quiet or -q is specified. Add tests for --quiet. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert push to builtin Add stash push to the helper. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert create to builtin Add stash create to the helper. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert store to builtin Add stash store to the helper and delete the store_stash function from the shell script. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert show to builtin Add stash show to the helper and delete the show_stash, have_stash, assert_stash_like, is_stash_like and parse_flags_and_rev functions from the shell script now that they are no longer needed. In shell version, although git stash show accepts --index and --quiet options, it ignores them. In C, both options are passed further to git diff. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert list to builtin Add stash list to the helper and delete the list_stash function from the shell script. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: convert pop to builtin Add stash pop to the helper and delete the pop_stash, drop_stash, assert_stash_ref functions from the shell script now that they are no longer needed. Signed-off-by: Joel Teichroeb [email protected] Signed-off-by: Paul-Sebastian Ungureanu [email protected] by klusark
🛠️ stash: convert branch to builtin Add stash branch to the helper and delete the apply_to_branch function from the shell script. Checkout does not currently provide a function for checking out a branch as cmd_checkout does a large amount of sanity checks first that we require here. Signed-off-by: Joel Teichroeb [email protected] Signed-off-by: Paul-Sebastian Ungureanu [email protected] by klusark
🛠️ stash: convert drop and clear to builtin Add the drop and clear commands to the builtin helper. These two are each simple, but are being added together as they are quite related. We have to unfortunately keep the drop and clear functions in the shell script as functions are called with parameters internally that are not valid when the commands are called externally. Once pop is converted they can both be removed. Signed-off-by: Joel Teichroeb [email protected] Signed-off-by: Paul-Sebastian Ungureanu [email protected] by klusark
🛠️ stash: convert apply to builtin Add a builtin helper for performing stash commands. Converting all at once proved hard to review, so starting with just apply lets conversion get started without the other commands being finished. The helper is being implemented as a drop in replacement for stash so that when it is complete it can simply be renamed and the shell script deleted. Delete the contents of the apply_stash shell function and replace it with a call to stash--helper apply until pop is also converted. Signed-off-by: Joel Teichroeb [email protected] Signed-off-by: Paul-Sebastian Ungureanu [email protected] by klusark
🛠️ fixup! Merge 'ps/stash-in-c' Let's drop an older version of this, in preparation for merging a newer version in the next commit. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ stash: mention options in show synopsis Mention in the documentation, that show accepts any option known to git diff. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: add tests for git stash show config This commit introduces tests for git stash show config. It tests all the cases where stash.showStat and stash.showPatch are unset or set to true / false. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ stash: rename test cases to be more descriptive Rename some test cases' labels to be more descriptive and under 80 characters per line. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ t3903: modernize style Remove whitespaces after redirection operators and wrap long lines. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by dscho
🛠️ stash: improve option parsing test coverage In preparation for converting the stash command incrementally to a builtin command, this patch improves test coverage of the option parsing. Both for having too many parameters, or too few. Signed-off-by: Joel Teichroeb [email protected] Signed-off-by: Paul-Sebastian Ungureanu [email protected] by klusark
🛠️ fixup! strbuf.c: add strbuf_join_argv() by ungps
🛠️ strbuf.c: add strbuf_join_argv() Implement strbuf_join_argv() to join arguments into a strbuf. Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ sha1-name.c: add get_oidf() which acts like get_oid() Compared to get_oid(), get_oidf() has as parameters a pointer to object_id, a printf format string and additional arguments. This will help simplify the code in subsequent commits. Original-idea-by: Johannes Schindelin [email protected] Signed-off-by: Paul-Sebastian Ungureanu [email protected] by ungps
🛠️ Merge branch 'mingw-update-msdn-link-v1' by dscho
🛠️ Merge branch 'rebase-r-and-merge-head-v1' by dscho
🛠️ mingw: replace an obsolete link with the superseding one The MSDN documentation has been superseded by Microsoft Docs (which is backed by a repository on GitHub containing many, many files in Markdown format). Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ status: rebase and merge can be in progress at the same time Since git rebase -r was introduced, that is possible. But our machinery did not think that possible, and failed to say anything about the rebase in progress when in the middle of a merge. Let's work around that in the minimal fashion. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ built-in rebase --skip/--abort: clean up stale .git/ files The scripted version of the rebase used to execute git reset --hard when skipping or aborting. When we ported this to C, we did update the worktree and some reflogs, but we failed to imitate git reset --hard's behavior regarding files in .git/ such as MERGE_HEAD. Let's address this oversight. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase -i: include MERGE_HEAD into files to clean up Every once in a while, the interactive rebase makes sure that no stale files are lying around. These days, we need to include MERGE_HEAD into that set of files, as the merge command will generate them. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase -r: do not write MERGE_HEAD unless needed When we detect that a merge can be skipped because the merged commit is already an ancestor of HEAD, we do not need to commit, therefore writing the MERGE_HEAD file is useless. It is actually worse than useless: a subsequent git commit will pick it up and think that we want to merge that commit, still. To avoid that, move the code that writes the MERGE_HEAD file to a location where we already know that the merge cannot be skipped. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ rebase -r: demonstrate bug with conflicting merges When calling merge on a branch that has already been merged, that merge is skipped quietly, but currently a MERGE_HEAD file is being left behind and will then be grabbed by the next pick (that did not want to create a merge commit). Demonstrate this. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Win32: Make the dirent implementation pluggable Emulating the POSIX dirent API on Windows via FindFirstFile/FindNextFile is pretty staightforward, however, most of the information provided in the WIN32_FIND_DATA structure is thrown away in the process. A more sophisticated implementation may cache this data, e.g. for later reuse in calls to lstat. Make the dirent implementation pluggable so that it can be switched at runtime, e.g. based on a config option. Define a base DIR structure with pointers to readdir/closedir that match the opendir implementation (i.e. similar to vtable pointers in OOP). Define readdir/closedir so that they call the function pointers in the DIR structure. This allows to choose the opendir implementation on a call-by-call basis. Move the fixed sized dirent.d_name buffer to the dirent-specific DIR structure, as d_name may be implementation specific (e.g. a caching implementation may just set d_name to point into the cache instead of copying the entire file name string). Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: dirent.c: Move opendir down Move opendir down in preparation for the next patch. Signed-off-by: Karsten Blees [email protected] by kblees
🛠️ Win32: make FILETIME conversion functions public We will use them in the upcoming "FSCache" patches (to accelerate sequential lstat() calls). Signed-off-by: Karsten Blees [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge branch 'visual-studio' Signed-off-by: Johannes Schindelin [email protected] by jeffhostetler
🛠️ Merge branch 'msvc' Signed-off-by: Johannes Schindelin [email protected] by jeffhostetler
🛠️ git: avoid calling aliased builtins via their dashed form This is one of the few places where Git violates its own deprecation of the dashed form. It is not necessary, either. As of 595d59e2b53 (git.c: ignore pager.* when launching builtin as dashed external, 2017-08-02), Git wants to ignore the pager.* config setting when expanding aliases. So let's strip out the check_pager_config() call from the copy-edited code. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ t5505,t5516: create .git/branches/ when needed It is a real old anachronism from the Cogito days to have a .git/branches/ directory. And to have tests that ensure that Cogito users can migrate away from using that directory. But so be it, let's continue testing it. Let's make sure, however, that git init does not need to create that directory. This bug was noticed when testing with templates that had been pre-committed, skipping the empty branches/ directory of course because Git does not track empty directories. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ bin-wrappers: append .exe to target paths if necessary When compiling with Visual Studio, the projects' names are identical to the executables modulo the extensions. Read: there will exist both a directory called git as well as an executable called git.exe in the end. Which means that the bin-wrappers need to target the .exe files lest they try to execute directories. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ .gitignore: ignore Visual Studio's temporary/generated files Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ WIP .gitignore: ignore library directories created by MSVC VS2008 buildsystem TODO: test whether we can drop this. Signed-off-by: Philip Oakley [email protected] by PhilipOakley
🛠️ .gitignore: touch up the entries regarding Visual Studio Add the Microsoft .manifest pattern, and do not anchor the 'Debug' and 'Release' entries at the top-level directory, to allow for multiple projects (one per target). Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ vcxproj: also link-or-copy builtins The problem with not having, say, git-receive-pack.exe after a full build is that the test suite will then happily use the installed git-receive-pack.exe because it finds nothing else. Absolutely not what we want. We want to have confidence that our test covers the MSVC-built Git executables, and not some random stuff. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ msvc: add a Makefile target to pre-generate the VS solution The entire idea of generating the VS solution makes only sense if we generate it via Continuous Integration; otherwise potential users would still have to download the entire Git for Windows SDK. So let's just add a target in the Makefile that can be used to generate said solution; The generated files will then be committed so that they can be pushed to a branch ready to check out by Visual Studio users. To make things even more useful, we also generate and commit other files that are required to run the test suite, such as templates and bin-wrappers: with this, developers can run the test suite in a regular Git Bash (that is part of a regular Git for Windows installation) after building the solution in Visual Studio. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ contrib/buildsystems: add a backend for modern Visual Studio versions Based on the previous patch series to be able to compile Git using Visual C++ from the command-line, this patch offers to generate project definitions for Visual Studio, so that Git can be developed in a modern IDE. Based on the generator for Visual Studio versions <= 2008 (which used .sln/.vcproj files), this patch copy-edits the generator of the .vcproj files to a new generator that produces .vcxproj files ready for Visual Studio 2010 and later (or MSBuild). As the vcpkg system (which is used to build Git's dependencies) cannot run in parallel (it does not lock, wreaking havoc with files being accessed and written at the same time, letting the vcpkg processes stumble over each others' toes), we make libgit the root of the project dependency tree and initialize the vcpkg system in this project's PreBuildEvent. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ contrib/buildsystems: error out on unknown option One time too many did this developer call the generate script passing a --make-out=<PATH> option that was happily ignored (because there should be a space, not an equal sign, between --make-out and the path). And one time too many, this script not only ignored it but did not even complain. Let's fix that. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ contrib/buildsystems: handle options starting with a slash With the recent changes to allow building with MSVC=1, we now pass the /OPT:REF option to the compiler. This confuses the parser that wants to turn the output of a dry run into project definitions for QMake and Visual Studio: Unhandled link option @ line 213: /OPT:REF at [...] Let's just extend the code that passes through options that start with a dash, so that it passes through options that start with a slash, too. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ contrib/buildsystems: handle libiconv, too Git's test suite shows tons of breakages unless Git is compiled without NO_ICONV. That means, in turn, that we need to generate build definitions with libiconv, which in turn implies that we have to handle the -liconv option properly. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ contrib/buildsystems: handle the curl library option Upon seeing the '-lcurl' option, point to the libcurl.lib. While there, fix the elsif indentation. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ contrib/buildsystems: optionally capture the dry-run in a file Add an option for capturing the output of the make dry-run used in determining the msvc-build structure for easy debugging. You can use the output of --make-out <path> in subsequent runs via the --in <path> option. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ contrib/buildsystems: redirect errors of the dry run into a log file Rather than swallowing the errors, it is better to have them in a file. To make it obvious what this is about, use the file name 'msvc-build-makedryerrors.txt'. Further, if the output is empty, simply delete that file. As we target Git for Windows' SDK (which, unlike its predecessor msysGit, offers Perl versions newer than 5.8), we can use the quite readable syntax if -f -z $ErrsFile (available in Perl >=5.10). Note that the file will contain the new values of the GIT_VERSION and GITGUI_VERSION if they were generated by the make file. They are omitted if the release is tagged and indentically defined in their respective GIT_VERSION_GEN file DEF_VER variables. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ contrib/buildsystems: ignore gettext stuff Git's build contains steps to handle internationalization. This caused hiccups in the parser used to generate QMake/Visual Studio project files. As those steps are irrelevant in this context, let's just ignore them. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ contrib/buildsystems: handle quoted spaces in filenames The engine.pl script expects file names not to contain spaces. However, paths with spaces are quite prevalent on Windows. Use shellwords() rather than split() to parse them correctly. Helped-by: Junio C Hamano [email protected] Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ contrib/buildsystems: fix misleading error message The error message talked about a "lib option", but it clearly referred to a link option. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ contrib/buildsystems: ignore irrelevant files in Generators/ The Generators/ directory can contain spurious files such as editors' backup files. Even worse, there could be .swp files which are not even valid Perl scripts. Let's just ignore anything but .pm files in said directory. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ msvc: ignore .dll and incremental compile output Ignore .dll files copied into the top-level directory. Ignore MSVC incremental compiler output files. Signed-off-by: Jeff Hostetler [email protected] Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ contrib/buildsystems: ignore invalidcontinue.obj Since 4b623d8 (MSVC: link in invalidcontinue.obj for better POSIX compatibility, 2014-03-29), invalidcontinue.obj is linked in the MSVC build, but it was not parsed correctly by the buildsystem. Ignore it, as it is known to Visual Studio and will be handled elsewhere. Also only substitute filenames ending with .o when generating the source .c filename, otherwise we would start to expect .cbj files to generate .obj files (which are not generated by our build)... In the future there may be source files that produce .obj files so keep the two issues (.obj files with & without source files) separate. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Duncan Smart [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ msvc: avoid debug assertion windows in Debug Mode For regular debugging, it is pretty helpful when a debug assertion in a running application triggers a window that offers to start the debugger. However, when running the test suite, it is not so helpful, in particular when the debug assertions are then suppressed anyway because we disable the invalid parameter checking (via invalidcontinue.obj, see the comment in config.mak.uname about that object for more information). So let's simply disable that window in Debug Mode (it is already disabled in Release Mode). Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Vcproj.pm: urlencode '<' and '>' when generating VC projects Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Vcproj.pm: do not configure VCWebServiceProxyGeneratorTool It is not necessary, and Visual Studio 2015 no longer supports it, anyway. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Vcproj.pm: list git.exe first to be startup project Visual Studio takes the first listed application/library as the default startup project [1]. Detect the 'git' project and place it the head of the apps list, rather than the tail. Export the apps list before libs list for both the projects and global structures of the .sln file. [1] http://stackoverflow.com/questions/1238553/ vs2008-where-is-the-startup-project-setting-stored-for-a-solution "In the solution file, there are a list of pseudo-XML "Project" entries. It turns out that whatever is the first one ends up as the Startup Project, unless it’s overridden in the suo file. Argh. I just rearranged the order in the file and it’s good." "just moving the pseudo-xml isn't enough. You also have to move the group of entries in the "GlobalSection(ProjectConfigurationPlatforms) = postSolution" group that has the GUID of the project you moved to the top. So there are two places to move lines." Signed-off-by: Philip Oakley [email protected] by PhilipOakley
🛠️ Vcproj.pm: auto-generate GUIDs We ran out GUIDs. Again. But there is no need to: we can generate them semi-randomly from the target file name of the project. Note: the Vcproj generator is probably only interesting for historical reasons; nevertheless, the upcoming Vcxproj generator (to support modern Visual Studio versions) is based on the Vcproj generator and it is better to fix this here first. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ msvc: support building Git using MS Visual C++ With this patch, Git can be built using the Microsoft toolchain, via: make MSVC=1 [DEBUG=1] Third party libraries are built from source using the open source "vcpkg" tool set. See https://github.com/Microsoft/vcpkg On a first build, the vcpkg tools and the third party libraries are automatically downloaded and built. DLLs for the third party libraries are copied to the top-level (and t/helper) directory to facilitate debugging. See compat/vcbuild/README. A series of .bat files are invoked by the Makefile to find the location of the installed version of Visual Studio and the associated compiler tools (essentially replicating the environment setup performed by a "Developer Command Prompt"). This should find the most recent VS2015 or VS2017 installation. Output from these scripts are used by the Makefile to define compiler and linker pathnames and -I and -L arguments. The build produces .pdb files for both debug and release builds. Note: This commit was squashed from an organic series of commits developed between 2016 and 2018 in Git for Windows' master branch. This combined commit eliminates the obsolete commits related to fetching NuGet packages for third party libraries. It is difficult to use NuGet packages for C/C++ sources because they may be built by earlier versions of the MSVC compiler and have CRT version and linking issues. Additionally, the C/C++ NuGet packages that were using tended to not be updated concurrently with the sources. And in the case of cURL and OpenSSL, this could expose us to security issues. Helped-by: Yue Lin Ho [email protected] Helped-by: Philip Oakley [email protected] Signed-off-by: Jeff Hostetler [email protected] Signed-off-by: Johannes Schindelin [email protected] by jeffhostetler
🛠️ msvc: do not pretend to support all signals This special-cases various signals that are not supported on Windows, such as SIGPIPE. These cause the UCRT to throw asserts (at least in debug mode). Signed-off-by: Jeff Hostetler [email protected] by jeffhostetler
🛠️ msvc: add pragmas for common warnings MSVC can be overzealous about some warnings. Disable them. Signed-off-by: Philip Oakley [email protected] by PhilipOakley
🛠️ msvc: fix detect_msys_tty() The ntstatus.h header is only available in MINGW. Signed-off-by: Jeff Hostetler [email protected] by jeffhostetler
🛠️ msvc: define ftello() It is just called differently in MSVC's headers. Signed-off-by: Jeff Hostetler [email protected] Signed-off-by: Johannes Schindelin [email protected] by jeffhostetler
🛠️ msvc: do not re-declare the timespec struct VS2015's headers already declare that struct. Signed-off-by: Jeff Hostetler [email protected] by jeffhostetler
🛠️ msvc: mark a variable as non-const VS2015 complains when using a const pointer in memcpy()/free(). Signed-off-by: Jeff Hostetler [email protected] by jeffhostetler
🛠️ msvc: define O_ACCMODE This constant is not defined in MSVC's headers. In UCRT's fcntl.h, _O_RDONLY, _O_WRONLY and _O_RDWR are defined as 0, 1 and 2, respectively. Yes, that means that UCRT breaks with the tradition that O_RDWR == O_RDONLY | O_WRONLY. It is a perfectly legal way to define those constants, though, therefore we need to take care of defining O_ACCMODE accordingly. This is particularly important in order to keep our "open() can set errno to EISDIR" emulation working: it tests that (flags & O_ACCMODE) is not identical to O_RDONLY before going on to test specifically whether the file for which open() reported EACCES is, in fact, a directory. Signed-off-by: Philip Oakley [email protected] Signed-off-by: Johannes Schindelin [email protected] by PhilipOakley
🛠️ msvc: include sigset_t definition On MSVC (VS2008) sigset_t is not defined. Signed-off-by: Philip Oakley [email protected] by PhilipOakley
🛠️ msvc: fix dependencies of compat/msvc.c The file compat/msvc.c includes compat/mingw.c, which means that we have to recompile compat/msvc.o if compat/mingw.c changes. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ mingw: replace mingw_startup() hack Git for Windows has special code to retrieve the command-line parameters (and even the environment) in UTF-16 encoding, so that they can be converted to UTF-8. This is necessary because Git for Windows wants to use UTF-8 encoded strings throughout its code, and the main() function does not get the parameters in that encoding. To do that, we used the __wgetmainargs() function, which is not even a Win32 API function, but provided by the MINGW "runtime" instead. Obviously, this method would not work with any other compiler than GCC, and in preparation for compiling with Visual C++, we would like to avoid that. Lucky us, there is a much more elegant way: we simply implement wmain() and link with -municode. The command-line parameters are passed to wmain() encoded in UTF-16, as desired, and this method also works with Visual C++. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ obstack: fix compiler warning MS Visual C suggests that the construct condition ? (int) i : (ptrdiff_t) d is incorrect. Let's fix this by casting to ptrdiff_t also for the positive arm of the conditional. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ cache-tree.c: avoid reusing the DEBUG constant In MSVC, the DEBUG constant is set automatically whenever compiling with debug information. This is clearly not what was intended in cache-tree.c, so let's use a less ambiguous constant there. Signed-off-by: Jeff Hostetler [email protected] by jeffhostetler
🛠️ t0001 (mingw): do not expect specific order of stdout/stderr When redirecting stdout/stderr to the same file, we cannot guarantee that stdout will come first. In fact, in this test case, it seems that an MSVC build always prints stderr first. In any case, this test case does not want to verify the order but the presence of both outputs, so let's relax the test a little. Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Mark .bat files as requiring CR/LF endings Signed-off-by: Johannes Schindelin [email protected] by dscho
🛠️ Merge pull request #1900 from tanushree27/remove-ipv6-fallback [Outreachy] Removed ipv6 fallback by dscho
🛠️ mingw: remove obsolete IPv6-related code To support IPv6, Git provided fall back functions for Windows versions that did not support IPv6. However, as Git dropped support for Windows XP and prior, those functions are not needed anymore. Removed those fallbacks by reverting commit[1] and using the functions directly (without 'ipv6_' prefix). [1] fe3b2b7b827c75c21d61933e073050b6840f6dbc. Signed-off-by: tanushree27 [email protected] by tanushree27

---

CONTRIBUTORS

Last week there were 14 contributors.
👤 dscho
👤 peff
👤 benpeart
👤 jeffhostetler
👤 kblees
👤 sschuberth
👤 nalla
👤 cesarb
👤 vangdfang
👤 gitster
👤 ungps
👤 klusark
👤 PhilipOakley
👤 tanushree27

---

STARGAZERS

Last week there were 16 stagazers.
⭐ baby5
⭐ vicoron
⭐ sszth
⭐ chenzx
⭐ Jayr2trill
⭐ mylesmarkevich
⭐ kvid
⭐ idealvin
⭐ navizal
⭐ YourReligion
⭐ quroni
⭐ zeroone2numeral2
⭐ tomsonTang
⭐ ylxb2016
⭐ tjuerge
⭐ bearmingo
You all are the stars! 🌟

---

RELEASES

Last week there were no releases.

---

That's all for last week, please 👀 Watch and ⭐ Star the repository git-for-windows/git to receive next weekly updates. 😃

You can also view all Weekly Digests by clicking here.

Your Weekly Digest bot. 📆