Improve git clone #5507

star-hengxing · 2024-08-26T07:30:35Z

Is your feature request related to a problem? Please describe.

Before:

git clone https://github.com/EsotericSoftware/spine-runtimes.git -c core.longpaths=true -c core.fsmonitor=false source.tmp\spine-runtimes
Cloning into 'source.tmp\spine-runtimes'...
remote: Enumerating objects: 120521, done.
remote: Counting objects: 100% (8354/8354), done.
remote: Compressing objects: 100% (3149/3149), done.
remote: Total 120521 (delta 5148), reused 7800 (delta 4715), pack-reused 112167 (from 1)
Receiving objects: 100% (120521/120521), 466.56 MiB | 16.85 MiB/s, done.
Resolving deltas: 100% (82459/82459), done.
Updating files: 100% (5541/5541), done.
git -c core.fsmonitor=false checkout d33c10f85634d01efbe4a3ab31dabaeaca41230c
Updating files: 100% (5871/5871), done.

Now:

Add --filter=tree:0 --no-checkout

git clone https://github.com/EsotericSoftware/spine-runtimes.git -c core.longpaths=true -c core.fsmonitor=false --filter=tree:0 --no-checkout source.tmp\spine-runtimes
Cloning into 'source.tmp\spine-runtimes'...
remote: Enumerating objects: 9495, done.
remote: Counting objects: 100% (580/580), done.
remote: Compressing objects: 100% (569/569), done.
remote: Total 9495 (delta 11), reused 571 (delta 11), pack-reused 8915 (from 1)
Receiving objects: 100% (9495/9495), 1.91 MiB | 7.91 MiB/s, done.
Resolving deltas: 100% (176/176), done.
git -c core.fsmonitor=false checkout d33c10f85634d01efbe4a3ab31dabaeaca41230c
remote: Enumerating objects: 703, done.
remote: Counting objects: 100% (507/507), done.
remote: Compressing objects: 100% (422/422), done.
remote: Total 703 (delta 26), reused 308 (delta 21), pack-reused 196 (from 1)
Receiving objects: 100% (703/703), 185.71 KiB | 2.00 MiB/s, done.
Resolving deltas: 100% (27/27), done.
remote: Enumerating objects: 496, done.
remote: Counting objects: 100% (183/183), done.
remote: Compressing objects: 100% (182/182), done.
remote: Total 496 (delta 3), reused 1 (delta 1), pack-reused 313 (from 1)
Receiving objects: 100% (496/496), 136.90 KiB | 1.63 MiB/s, done.
Resolving deltas: 100% (6/6), done.
remote: Enumerating objects: 3505, done.
remote: Counting objects: 100% (1488/1488), done.
remote: Compressing objects: 100% (1121/1121), done.
remote: Total 3505 (delta 908), reused 367 (delta 367), pack-reused 2017 (from 1)
Receiving objects: 100% (3505/3505), 51.29 MiB | 16.72 MiB/s, done.
Resolving deltas: 100% (1491/1491), done.
Updating files: 100% (3907/3907), done.

Describe the solution you'd like

https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone

Treeless clones are really only helpful for automated builds when you want to quickly clone, compile a project, then throw away the repository. In environments like GitHub Actions using public runners, you want to minimize your clone time so you can spend your machine time actually building your software! Treeless clones might be an excellent option for those environments.

Describe alternatives you've considered

No response

Additional context

No response

The text was updated successfully, but these errors were encountered:

waruqi · 2024-08-26T09:00:34Z

这两参数的兼容性怎样，低版本 git 支持么

Issues-translate-bot · 2024-08-26T09:00:45Z

Bot detected the issue body's language is not English, translate it automatically.

What is the compatibility of these two parameters? Is it supported by lower versions of git?

waruqi · 2024-08-26T09:03:05Z

另外，目前 clone 需要指定某个 branch，git clone --depth 1 -b branchname 这种支持么，depth 还需要指定么

Issues-translate-bot · 2024-08-26T09:03:17Z

Bot detected the issue body's language is not English, translate it automatically.

In addition, currently clone needs to specify a certain branch. Is git clone --depth 1 -b branchname supported? Does depth still need to be specified?

star-hengxing · 2024-08-26T09:12:56Z

这两参数的兼容性怎样，低版本 git 支持么

暂时没找到哪个版本给的，但看 partial-clone 里面的时间，大概 2021 年就有了。

另外，目前 clone 需要指定某个 branch，git clone --depth 1 -b branchname 这种支持么，depth 还需要指定么

看 https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone ，这个就是 depth 1 加强版。

waruqi · 2024-08-26T09:27:16Z

快是快，但是 clone 下来没啥东西么。。

ruki-2:tmp ruki$ git clone --filter=tree:0 --no-checkout [email protected]:xmake-io/xmake.git 
Cloning into 'xmake'...
remote: Enumerating objects: 14502, done.
remote: Counting objects: 100% (370/370), done.
remote: Compressing objects: 100% (363/363), done.
remote: Total 14502 (delta 7), reused 330 (delta 7), pack-reused 14132 (from 1)
Receiving objects: 100% (14502/14502), 4.76 MiB | 704.00 KiB/s, done.
Resolving deltas: 100% (44/44), done.
ruki-2:tmp ruki$ cd xmake
ruki-2:xmake ruki$ ls
ruki-2:xmake ruki$ ls -a
.	..	.git

SirLynix · 2024-08-26T09:33:16Z

that's because of the --no-checkout, not --filter=tree:0

star-hengxing · 2024-08-26T09:39:58Z

Useful for package that have git url(imgui docking/spine-runtimes/godotcpp4 etc...)

waruqi · 2024-08-26T09:43:48Z

https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone

it works.

waruqi · 2024-08-26T09:44:13Z

On github.com and GitHub Enterprise Server 2.22+, there are two options available:
Treeless clones: git clone --filter=tree:0

Does all git servers support it?

waruqi · 2024-08-26T09:47:50Z

#5509

waruqi · 2024-08-26T12:40:08Z

try it again.

waruqi · 2024-08-26T12:46:43Z

测试了下，似乎 depth 1 更快么。

depth 1

[ruki@5dd706d1f15e xmake-repo]$ time xmake update -s -v dev
checking for unzip ... /usr/sbin/unzip
checking for git ... /usr/sbin/git
checking for gzip ... /usr/sbin/gzip
checking for tar ... /usr/sbin/tar
checking for ping ... no
/usr/sbin/git ls-remote --refs https://github.com/xmake-io/xmake.git
update version dev from official source ..
prepared to download to temp dir /tmp/.xmake971/240826/xmakesrc/dev ..
  => downloading https://github.com/xmake-io/xmake.git .. /usr/sbin/git clone https://github.com/xmake-io/xmake.git -b dev --depth 1 -c core.fsmonitor=false /tmp/.xmake971/240826/xmakesrc/dev
Cloning into '/tmp/.xmake971/240826/xmakesrc/dev'...
remote: Enumerating objects: 4233, done.
remote: Counting objects: 100% (4233/4233), done.
remote: Compressing objects: 100% (3288/3288), done.
remote: Total 4233 (delta 1220), reused 2351 (delta 448), pack-reused 0 (from 0)
Receiving objects: 100% (4233/4233), 2.75 MiB | 2.62 MiB/s, done.
Resolving deltas: 100% (1220/1220), done.
  => download https://github.com/xmake-io/xmake.git .. ok
  => install script to /home/ruki/.local/share/xmake .. sh /home/ruki/.local/share/xmake/scripts/update-script.sh /home/ruki/.local/share/xmake /tmp/.xmake971/240826/xmakesrc/dev/xmake
ok

real	0m4.580s
user	0m0.587s
sys	0m0.806s

treeless

[ruki@5dd706d1f15e xmake-repo]$ time xmake update -s -v dev
checking for unzip ... /usr/sbin/unzip
checking for git ... /usr/sbin/git
checking for gzip ... /usr/sbin/gzip
checking for tar ... /usr/sbin/tar
checking for ping ... no
/usr/sbin/git ls-remote --refs https://github.com/xmake-io/xmake.git
update version dev from official source ..
prepared to download to temp dir /tmp/.xmake971/240826/xmakesrc/dev ..
  => downloading https://github.com/xmake-io/xmake.git .. /usr/sbin/git clone https://github.com/xmake-io/xmake.git -b dev --filter=tree:0 -c core.fsmonitor=false /tmp/.xmake971/240826/xmakesrc/dev
Cloning into '/tmp/.xmake971/240826/xmakesrc/dev'...
remote: Enumerating objects: 14506, done.
remote: Counting objects: 100% (374/374), done.
remote: Compressing objects: 100% (367/367), done.
remote: Total 14506 (delta 7), reused 333 (delta 7), pack-reused 14132 (from 1)
Receiving objects: 100% (14506/14506), 4.76 MiB | 3.02 MiB/s, done.
Resolving deltas: 100% (44/44), done.
remote: Enumerating objects: 1349, done.
remote: Counting objects: 100% (322/322), done.
remote: Compressing objects: 100% (288/288), done.
remote: Total 1349 (delta 7), reused 111 (delta 4), pack-reused 1027 (from 1)
Receiving objects: 100% (1349/1349), 160.39 KiB | 423.00 KiB/s, done.
Resolving deltas: 100% (15/15), done.
remote: Enumerating objects: 2883, done.
remote: Counting objects: 100% (2010/2010), done.
remote: Compressing objects: 100% (1720/1720), done.
remote: Total 2883 (delta 1018), reused 355 (delta 289), pack-reused 873 (from 1)
Receiving objects: 100% (2883/2883), 2.61 MiB | 2.20 MiB/s, done.
Resolving deltas: 100% (1168/1168), done.
Updating files: 100% (3494/3494), done.
  => download https://github.com/xmake-io/xmake.git .. ok
  => install script to /home/ruki/.local/share/xmake .. sh /home/ruki/.local/share/xmake/scripts/update-script.sh /home/ruki/.local/share/xmake /tmp/.xmake971/240826/xmakesrc/dev/xmake
ok

real	0m9.093s
user	0m0.932s
sys	0m1.501s

waruqi · 2024-08-26T12:50:34Z

depth 1

[ruki@5dd706d1f15e tmp]$ time git clone https://github.com/EsotericSoftware/spine-runtimes.git --depth 1
Cloning into 'spine-runtimes'...
remote: Enumerating objects: 5367, done.
remote: Counting objects: 100% (5367/5367), done.
remote: Compressing objects: 100% (3886/3886), done.
remote: Total 5367 (delta 1939), reused 3895 (delta 1351), pack-reused 0 (from 0)
Receiving objects: 100% (5367/5367), 81.73 MiB | 3.24 MiB/s, done.
Resolving deltas: 100% (1939/1939), done.
Updating files: 100% (5541/5541), done.

real	0m29.297s
user	0m3.782s
sys	0m7.461s

treeless

[ruki@5dd706d1f15e tmp]$ time git clone https://github.com/EsotericSoftware/spine-runtimes.git --filter=tree:0
Cloning into 'spine-runtimes'...
remote: Enumerating objects: 9496, done.
remote: Counting objects: 100% (581/581), done.
remote: Compressing objects: 100% (570/570), done.
remote: Total 9496 (delta 11), reused 571 (delta 11), pack-reused 8915 (from 1)
Receiving objects: 100% (9496/9496), 1.91 MiB | 1.93 MiB/s, done.
Resolving deltas: 100% (185/185), done.
remote: Enumerating objects: 703, done.
remote: Counting objects: 100% (507/507), done.
remote: Compressing objects: 100% (423/423), done.
remote: Total 703 (delta 25), reused 307 (delta 20), pack-reused 196 (from 1)
Receiving objects: 100% (703/703), 186.42 KiB | 763.00 KiB/s, done.
Resolving deltas: 100% (26/26), done.
remote: Enumerating objects: 4663, done.
remote: Counting objects: 100% (2417/2417), done.
remote: Compressing objects: 100% (1682/1682), done.
remote: Total 4663 (delta 1191), reused 1193 (delta 731), pack-reused 2246 (from 1)
Receiving objects: 100% (4663/4663), 81.45 MiB | 2.81 MiB/s, done.
Resolving deltas: 100% (1789/1789), done.
Updating files: 100% (5541/5541), done.

real	0m36.941s
user	0m4.327s
sys	0m8.773s

waruqi · 2024-08-26T12:56:53Z

I revert it first.

star-hengxing · 2024-08-26T13:02:56Z

Try xmake l scripts\test.lua --shallow -vD spine-runtimes

waruqi · 2024-08-26T13:05:36Z

Try xmake l scripts\test.lua --shallow -vD spine-runtimes

no package in xmake-repo

see

depth 1

[ruki@5dd706d1f15e tmp]$ time git clone https://github.com/EsotericSoftware/spine-runtimes.git --depth 1
Cloning into 'spine-runtimes'...
remote: Enumerating objects: 5367, done.
remote: Counting objects: 100% (5367/5367), done.
remote: Compressing objects: 100% (3886/3886), done.
remote: Total 5367 (delta 1939), reused 3895 (delta 1351), pack-reused 0 (from 0)
Receiving objects: 100% (5367/5367), 81.73 MiB | 3.24 MiB/s, done.
Resolving deltas: 100% (1939/1939), done.
Updating files: 100% (5541/5541), done.

real	0m29.297s
user	0m3.782s
sys	0m7.461s

treeless

[ruki@5dd706d1f15e tmp]$ time git clone https://github.com/EsotericSoftware/spine-runtimes.git --filter=tree:0
Cloning into 'spine-runtimes'...
remote: Enumerating objects: 9496, done.
remote: Counting objects: 100% (581/581), done.
remote: Compressing objects: 100% (570/570), done.
remote: Total 9496 (delta 11), reused 571 (delta 11), pack-reused 8915 (from 1)
Receiving objects: 100% (9496/9496), 1.91 MiB | 1.93 MiB/s, done.
Resolving deltas: 100% (185/185), done.
remote: Enumerating objects: 703, done.
remote: Counting objects: 100% (507/507), done.
remote: Compressing objects: 100% (423/423), done.
remote: Total 703 (delta 25), reused 307 (delta 20), pack-reused 196 (from 1)
Receiving objects: 100% (703/703), 186.42 KiB | 763.00 KiB/s, done.
Resolving deltas: 100% (26/26), done.
remote: Enumerating objects: 4663, done.
remote: Counting objects: 100% (2417/2417), done.
remote: Compressing objects: 100% (1682/1682), done.
remote: Total 4663 (delta 1191), reused 1193 (delta 731), pack-reused 2246 (from 1)
Receiving objects: 100% (4663/4663), 81.45 MiB | 2.81 MiB/s, done.
Resolving deltas: 100% (1789/1789), done.
Updating files: 100% (5541/5541), done.

real	0m36.941s
user	0m4.327s
sys	0m8.773s

star-hengxing · 2024-08-26T13:11:57Z

Update xrepo first xmake-io/xmake-repo#5034

spine-runtimes need to checkout specify commit, so depth 1 not be useful on this case

waruqi · 2024-08-26T13:38:32Z

Update xrepo first xmake-io/xmake-repo#5034

spine-runtimes need to checkout specify commit, so depth 1 not be useful on this case

这个不能用来改进 git clone --depth 1，只能用来改进 git clone + git checkout 的 case。

#5510

star-hengxing · 2024-08-26T13:40:08Z

Useful for package that have git url(imgui docking/spine-runtimes/godotcpp4 etc...)

😥

waruqi · 2024-08-26T15:28:25Z

try it again.

Issues-translate-bot · 2024-08-26T16:06:19Z

Bot detected the issue body's language is not English, translate it automatically.

try it again.

star-hengxing added the feature request label Aug 26, 2024

waruqi mentioned this issue Aug 26, 2024

use treeless instead of depth1 #5509

Merged

waruqi added this to the v2.9.5 milestone Aug 26, 2024

waruqi closed this as completed Aug 26, 2024

waruqi added a commit that referenced this issue Aug 26, 2024

disable treeless for git #5507

5aeee42

waruqi mentioned this issue Aug 26, 2024

improve treeless for git.clone #5510

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve git clone #5507

Improve git clone #5507

star-hengxing commented Aug 26, 2024 •

edited

Loading

waruqi commented Aug 26, 2024

Issues-translate-bot commented Aug 26, 2024

waruqi commented Aug 26, 2024

Issues-translate-bot commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

SirLynix commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024 •

edited

Loading

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

depth 1

treeless

star-hengxing commented Aug 26, 2024 •

edited

Loading

waruqi commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

Issues-translate-bot commented Aug 26, 2024

Improve git clone #5507

Improve git clone #5507

Comments

star-hengxing commented Aug 26, 2024 • edited Loading

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

waruqi commented Aug 26, 2024

Issues-translate-bot commented Aug 26, 2024

waruqi commented Aug 26, 2024

Issues-translate-bot commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

SirLynix commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024 • edited Loading

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024

waruqi commented Aug 26, 2024

depth 1

treeless

waruqi commented Aug 26, 2024

depth 1

treeless

waruqi commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

depth 1

treeless

star-hengxing commented Aug 26, 2024 • edited Loading

waruqi commented Aug 26, 2024

star-hengxing commented Aug 26, 2024

waruqi commented Aug 26, 2024

Issues-translate-bot commented Aug 26, 2024

star-hengxing commented Aug 26, 2024 •

edited

Loading

waruqi commented Aug 26, 2024 •

edited

Loading

star-hengxing commented Aug 26, 2024 •

edited

Loading