Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tab completion is really slow #795

Closed
dwijnand opened this issue Mar 16, 2018 · 12 comments · Fixed by #960
Closed

tab completion is really slow #795

dwijnand opened this issue Mar 16, 2018 · 12 comments · Fixed by #960

Comments

@dwijnand
Copy link

in a clone of https://github.com/scala/scala run tig blame src/library/scala/math/Order<TAB> and (on my machine) it'll take 14 seconds to give you:

$ tig blame src/library/scala/math/Order
src/library/scala/math/Ordered.scala    src/library/scala/math/Ordering.scala

tig version 2.3.3
git version 2.15.1

@pzrq
Copy link

pzrq commented Mar 26, 2018

@dwijnand Great question. Could you run $ time tig blame src/library/scala/math/Order<TAB> and copy/paste the output here?

If you can think of any other way to debug, profile or otherwise get visibility into what is causing the slowness that would be great too.

@jonas
Copy link
Owner

jonas commented Mar 26, 2018 via email

@dwijnand
Copy link
Author

Could you run $ time tig blame src/library/scala/math/Order<TAB> and copy/paste the output here?

you can't time the bash completion - it just outputs the same thing:

$ time tig blame src/library/scala/math/Order
src/library/scala/math/Ordered.scala    src/library/scala/math/Ordering.scala

@pzrq
Copy link

pzrq commented Mar 27, 2018

Then is this a tig issue or a bash issue or some interaction of the two?

@dwijnand
Copy link
Author

it's an issue with tig's bash completion.

for comparison completing blame src/library/scala/math/Order<TAB> is less than a second for me.

@askedrelic
Copy link

I've noticed this as well on a git repo with 14K tags. I'm assuming tags are part of the autocomplete algorithm and that is what I assume really slows it down. I think it's more due to bash being slow at large amounts of data.

tig version 2.3.3
git version 2.16.3

@orehmane
Copy link

Just ran into this myself, with the following versions:

git version 2.20.1
tig version 2.4.1
ncurses version 5.7.20081102
readline version 7.0

The repo in question has 42468 tags, which I assume is the cause (since it's only happening in that one directory). It hangs seemingly indefinitely.

@ingydotnet
Copy link

On a huge repo I use daily, tig Bash completion hangs for 2 minutes. If I type tig and press tab, I'm screwed. Git completions on same repo are fast.

Here's a fun trick:

set -x; tig <tab><tab>

Try that on a big repo and sit back with some coffee and watch the show.

@rohieb
Copy link

rohieb commented Oct 17, 2019

The culprit is probably __tig_complete_file:

linux.git $ time __tig_complete_file 

real    0m29.363s
user    0m27.805s
sys     0m1.600s

linux.git $ time __git_complete_file

real    0m0.014s
user    0m0.003s
sys     0m0.008s

In the long run I would probably rewrite the tig completion so it uses the git completion for all git-related options. Most users should already have this installed, and it is faster in most cases, as well as better maintained :)

@rohieb
Copy link

rohieb commented Oct 17, 2019

linux.git $ time __tig_complete_file 

real    0m29.363s
user    0m27.805s
sys     0m1.600s

linux.git $ time __git_complete_file

real    0m0.014s
user    0m0.003s
sys     0m0.008s

These times are reproducible btw, so no caching issues.

rohieb added a commit to rohieb/tig that referenced this issue Oct 17, 2019
Fixes: jonas#795 ("tab completion is really slow")

When run on a Git repository with a large amount of files, like the
linux repository, __tig_complete_file() can take a very long time to
complete. This is unfortunate because when accidentally pressing Tab in
such a repo, bash gets stuck until the completion function has finished,
and does not even allow the user to cancel the operation with Ctrl-C.

In contrast, the current git completion does not have these problems,
and since tig's command line parameters are mostly parameters to git-log
or git-diff, we can use git's native completion for those cases instead.
This also has the advantage that we do not need to care about updating
the tig completion when new parameters are added to git-log or git-diff.

I have tested this only in bash, not on zsh.

For comparison, here is an exemplary runtime measurement of the old and
the new completion in bash, showing an improvement of factor 1863.
Admittedly, this was on a fairly loaded system (a build server), but
still then the new completion runs in unnoticable time. I'm also getting
similar results on an idle system in the same repo with runtime
improvements of about factor 1000.

    linux (master) $ echo $BASH_VERSION
    5.0.3(1)-release

    linux (master) $ git describe
    v5.4-rc3-38-gbc88f85c6c09

    linux (master) $ uptime
     16:45:52 up 36 days,  3:33, 224 users,  load average: 24.17, 38.87, 31.21

    # The old completion:
    linux (master) $ . /usr/share/bash-completion/completions/tig
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 _tig

    real    2m1.145s
    user    1m40.379s
    sys     0m1.347s

    # The new completion:
    linux (master) $ . ../tig/contrib/tig-completion.bash
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 __git_wrap_tig

    real    0m0.127s
    user    0m0.085s
    sys     0m0.024s

With this change, almost nothing of the old completion remains, so
change the copyright header accordingly. I'm also now adding a
GPL-2.0-or-later dedication, which is the same license as most other
code in this repository, and which, I presume, was also the author's
intent since the first incarnation of this file.

Signed-off-by: Roland Hieber <[email protected]>
rohieb added a commit to rohieb/tig that referenced this issue Oct 17, 2019
Fixes: jonas#795 ("tab completion is really slow")

When run on a Git repository with a large amount of files, like the
linux repository, __tig_complete_file() can take a very long time to
complete. This is unfortunate because when accidentally pressing Tab in
such a repo, bash gets stuck until the completion function has finished,
and does not even allow the user to cancel the operation with Ctrl-C.

In contrast, the current git completion does not have these problems,
and since tig's command line parameters are mostly parameters to git-log
or git-diff, we can use git's native completion for those cases instead.
This also has the advantage that we do not need to care about updating
the tig completion when new parameters are added to git-log or git-diff.

I have tested this only in bash, not on zsh.

For comparison, here is an exemplary runtime measurement of the old and
the new completion in bash, showing an improvement of factor 1863.
Admittedly, this was on a fairly loaded system (a build server), but
still then the new completion runs in unnoticable time. I'm also getting
similar results on an idle system in the same repo with runtime
improvements of about factor 1000.

    linux (master) $ echo $BASH_VERSION
    5.0.3(1)-release

    linux (master) $ git describe
    v5.4-rc3-38-gbc88f85c6c09

    linux (master) $ uptime
     16:45:52 up 36 days,  3:33, 224 users,  load average: 24.17, 38.87, 31.21

    # The old completion:
    linux (master) $ . /usr/share/bash-completion/completions/tig
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 _tig

    real    2m1.145s
    user    1m40.379s
    sys     0m1.347s

    # The new completion:
    linux (master) $ . ../tig/contrib/tig-completion.bash
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 __git_wrap_tig

    real    0m0.127s
    user    0m0.085s
    sys     0m0.024s

With this change, almost nothing of the old completion remains, so
change the copyright header accordingly. I'm also now adding a
GPL-2.0-or-later dedication, which is the same license as most other
code in this repository, and which, I presume, was also the author's
intent since the first incarnation of this file.

Signed-off-by: Roland Hieber <[email protected]>
@benpaxton-hf
Copy link

the loop in _tigcomp is the problem; I replaced it locally with an equivalent but significantly faster awk script before finding this issue and the PR - my tests showed that for the specific repo I was having issues with, it was taking 15+ seconds, iterating approximately 9000 times and building a 327kB string; the awk script took ~0.01s and built an identical string.

#960 is probably the "right" solution here, though.

rohieb added a commit to rohieb/tig that referenced this issue Feb 15, 2020
Fixes: jonas#795 ("tab completion is really slow")

When run on a Git repository with a large amount of files, like the
linux repository, __tig_complete_file() can take a very long time to
complete. This is unfortunate because when accidentally pressing Tab in
such a repo, bash gets stuck until the completion function has finished,
and does not even allow the user to cancel the operation with Ctrl-C.

In contrast, the current git completion does not have these problems,
and since tig's command line parameters are mostly parameters to git-log
or git-diff, we can use git's native completion for those cases instead.
This also has the advantage that we do not need to care about updating
the tig completion when new parameters are added to git-log or git-diff.

I have tested this only in bash, not in zsh.

For comparison, here is an exemplary runtime measurement of the old and
the new completion in bash, showing an improvement of factor 1863.
Admittedly, this was on a fairly loaded system (a build server), but
still then the new completion runs in unnoticable time. I'm also getting
similar results on an idle system in the same repo with runtime
improvements of about factor 1000.

    linux (master) $ echo $BASH_VERSION
    5.0.3(1)-release

    linux (master) $ git describe
    v5.4-rc3-38-gbc88f85c6c09

    linux (master) $ uptime
     16:45:52 up 36 days,  3:33, 224 users,  load average: 24.17, 38.87, 31.21

    # The new completion:
    linux (master) $ . ../tig/contrib/tig-completion.bash
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 __git_wrap_tig

    real    0m0.127s
    user    0m0.085s
    sys     0m0.024s

    # The old completion:
    linux (master) $ . /usr/share/bash-completion/completions/tig
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 _tig

    real    2m1.145s
    user    1m40.379s
    sys     0m1.347s

With this change, almost nothing of the old completion remains, so
change the copyright header accordingly. I'm also now adding a
GPL-2.0-or-later dedication, which is the same license as most other
code in this repository; and which, I presume, was also the author's
intent since the first incarnation of this file.

While at it, fix some typos in old comments, and update installation
instructions.

Signed-off-by: Roland Hieber <[email protected]>
rohieb added a commit to rohieb/tig that referenced this issue Feb 16, 2020
Fixes: jonas#795 ("tab completion is really slow")

When run on a Git repository with a large amount of files, like the
linux repository, __tig_complete_file() can take a very long time to
complete. This is unfortunate because when accidentally pressing Tab in
such a repo, bash gets stuck until the completion function has finished,
and does not even allow the user to cancel the operation with Ctrl-C.

In contrast, the current git completion does not have these problems,
and since tig's command line parameters are mostly parameters to git-log
or git-diff, we can use git's native completion for those cases instead.
This also has the advantage that we do not need to care about updating
the tig completion when new parameters are added to git-log or git-diff.

I have tested this only in bash, not in zsh.

For comparison, here is an exemplary runtime measurement of the old and
the new completion in bash, showing an improvement of factor 1863.
Admittedly, this was on a fairly loaded system (a build server), but
still then the new completion runs in unnoticable time. I'm also getting
similar results on an idle system in the same repo with runtime
improvements of about factor 1000.

    linux (master) $ echo $BASH_VERSION
    5.0.3(1)-release

    linux (master) $ git describe
    v5.4-rc3-38-gbc88f85c6c09

    linux (master) $ uptime
     16:45:52 up 36 days,  3:33, 224 users,  load average: 24.17, 38.87, 31.21

    # The new completion:
    linux (master) $ . ../tig/contrib/tig-completion.bash
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 __git_wrap_tig

    real    0m0.127s
    user    0m0.085s
    sys     0m0.024s

    # The old completion:
    linux (master) $ . /usr/share/bash-completion/completions/tig
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 _tig

    real    2m1.145s
    user    1m40.379s
    sys     0m1.347s

With this change, almost nothing of the old completion remains, so
change the copyright header accordingly. I'm also now adding a
GPL-2.0-or-later dedication, which is the same license as most other
code in this repository; and which, I presume, was also the author's
intent since the first incarnation of this file.

While at it, fix some typos in old comments, and update installation
instructions.

Signed-off-by: Roland Hieber <[email protected]>
rohieb added a commit to rohieb/tig that referenced this issue Feb 16, 2020
Fixes: jonas#795 ("tab completion is really slow")

When run on a Git repository with a large amount of files, like the
linux repository, __tig_complete_file() can take a very long time to
complete. This is unfortunate because when accidentally pressing Tab in
such a repo, bash gets stuck until the completion function has finished,
and does not even allow the user to cancel the operation with Ctrl-C.

In contrast, the current git completion does not have these problems,
and since tig's command line parameters are mostly parameters to git-log
or git-diff, we can use git's native completion for those cases instead.
This also has the advantage that we do not need to care about updating
the tig completion when new parameters are added to git-log or git-diff.

I have tested this only in bash, not in zsh.

For comparison, here is an exemplary runtime measurement of the old and
the new completion in bash, showing an improvement of factor 1863.
Admittedly, this was on a fairly loaded system (a build server), but
still then the new completion runs in unnoticable time. I'm also getting
similar results on an idle system in the same repo with runtime
improvements of about factor 1000.

    linux (master) $ echo $BASH_VERSION
    5.0.3(1)-release

    linux (master) $ git describe
    v5.4-rc3-38-gbc88f85c6c09

    linux (master) $ uptime
     16:45:52 up 36 days,  3:33, 224 users,  load average: 24.17, 38.87, 31.21

    # The new completion:
    linux (master) $ . ../tig/contrib/tig-completion.bash
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 __git_wrap_tig

    real    0m0.127s
    user    0m0.085s
    sys     0m0.024s

    # The old completion:
    linux (master) $ . /usr/share/bash-completion/completions/tig
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 _tig

    real    2m1.145s
    user    1m40.379s
    sys     0m1.347s

With this change, almost nothing of the old completion remains, so
change the copyright header accordingly. I'm also now adding a
GPL-2.0-or-later dedication, which is the same license as most other
code in this repository; and which, I presume, was also the author's
intent since the first incarnation of this file.

While at it, fix some typos in old comments, and update installation
instructions.

Signed-off-by: Roland Hieber <[email protected]>
koutcher pushed a commit that referenced this issue Feb 23, 2020
Fixes: #795 ("tab completion is really slow")

When run on a Git repository with a large amount of files, like the
linux repository, __tig_complete_file() can take a very long time to
complete. This is unfortunate because when accidentally pressing Tab in
such a repo, bash gets stuck until the completion function has finished,
and does not even allow the user to cancel the operation with Ctrl-C.

In contrast, the current git completion does not have these problems,
and since tig's command line parameters are mostly parameters to git-log
or git-diff, we can use git's native completion for those cases instead.
This also has the advantage that we do not need to care about updating
the tig completion when new parameters are added to git-log or git-diff.

I have tested this only in bash, not in zsh.

For comparison, here is an exemplary runtime measurement of the old and
the new completion in bash, showing an improvement of factor 1863.
Admittedly, this was on a fairly loaded system (a build server), but
still then the new completion runs in unnoticable time. I'm also getting
similar results on an idle system in the same repo with runtime
improvements of about factor 1000.

    linux (master) $ echo $BASH_VERSION
    5.0.3(1)-release

    linux (master) $ git describe
    v5.4-rc3-38-gbc88f85c6c09

    linux (master) $ uptime
     16:45:52 up 36 days,  3:33, 224 users,  load average: 24.17, 38.87, 31.21

    # The new completion:
    linux (master) $ . ../tig/contrib/tig-completion.bash
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 __git_wrap_tig

    real    0m0.127s
    user    0m0.085s
    sys     0m0.024s

    # The old completion:
    linux (master) $ . /usr/share/bash-completion/completions/tig
    linux (master) $ time COMP_WORDS=("tig log") COMP_CWORD=2 _tig

    real    2m1.145s
    user    1m40.379s
    sys     0m1.347s

With this change, almost nothing of the old completion remains, so
change the copyright header accordingly. I'm also now adding a
GPL-2.0-or-later dedication, which is the same license as most other
code in this repository; and which, I presume, was also the author's
intent since the first incarnation of this file.

While at it, fix some typos in old comments, and update installation
instructions.

Signed-off-by: Roland Hieber <[email protected]>
@dwijnand
Copy link
Author

Wonderful to have this fixed! Thank you, everyone involved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants