Feature request: Improve speed

[lervag]

Hi,

I have not looked at the code for vim-easy-align, so I am not sure to which extent it is optimized or not. However, when I use the :EasyAlign command for relatively complex tables that spans multiple lines and columns, I find it is rather slow. If there is any chance of optimizing the code for more speed, that would be great!

[junegunn]

Thanks for raising the issue. I'll see what I can do.

[lervag]

To give you an example to work with:

     32 & 1.14\e 1&     & 5.65\e 2&     & & & 1.16\e 1&     & 1.28\e 1&    \\
     64 & 1.03\e 1& 0.1 & 4.98\e 2& 0.2 & & & 9.21\e 2& 0.3 & 1.02\e 1& 0.3\\
    128 & 9.86\e 2& 0.1 & 4.69\e 2& 0.1 & & & 8.46\e 2& 0.1 & 9.45\e 2& 0.1\\
    256 & 9.65\e 2& 0.0 & 4.59\e 2& 0.0 & & & 8.15\e 2& 0.1 & 9.11\e 2& 0.1\\
    512 & 9.55\e 2& 0.0 & 4.56\e 2& 0.0 & & & 8.01\e 2& 0.0 & 8.96\e 2& 0.0\\
   1024 & 9.49\e 2& 0.0 & 4.53\e 2& 0.0 & & & 7.94\e 2& 0.0 & 8.89\e 2& 0.0\\
   2048 & 9.47\e 2& 0.0 & 4.52\e 2& 0.0 & & & 7.91\e 2& 0.0 & 8.85\e 2& 0.0\\
   4096 & 9.46\e 2& 0.0 & 4.51\e 2& 0.0 & & & 7.90\e 2& 0.0 & 8.83\e 2& 0.0\\
   8192 & 9.45\e 2& 0.0 & 4.51\e 2& 0.0 & & &         &     &         &    \\

To align this table with EasyAlign with default options and *& takes about 10 seconds on my small laptop. Granted, this is a rather slow laptop with intel core i5 processor. But I would think that the same example would still take quite a few seconds on a more powerful computer.

[junegunn]

Whoa, 10 seconds for just 9 lines? That is not normal.
It returns immediately on my humble mid-2011 MacBook Air with core 2 duo processor.
I was thinking that you were talking hundreds of lines.
Which version of Vim, and which OS are you running?

[lervag]

Ah, I see. I can confirm that if I use EasyAlign with no filetype, then it is fast here as well. The problem has to do with the combination of EasyAlign file type plugins for LaTeX (and perhaps other file types). I will investigate when I have the chance and try to find exactly what in the file type plugin that creates the lag.

[junegunn]

Hmm, can you try again with let g:easy_align_ignore_groups = []?
It disables syntax-aware alignment.

Finding out syntax highlighting group of a certain position is known to be a relatively slow operation. (:help synID)
Yet it's fast enough for me on my Vim. Maybe you're running an older version of Vim?

[lervag]

I tried that, but it did not seem to have any effect. However, I also tried to turn off folding, and then it seems to work as expected. I also tried a very coarse fix to easy_align#align(...):

  let oldfen = &fen
  let &fen = 0
  try
    call s:align(a:bang, a:firstline, a:lastline, a:expr)
  catch 'exit'
  endtry
  let &fen = oldfen

This also seems to fix my issue completely. I think this is the correct way to disable folding temporarily, but I am not sure if this is the correct place to add it in your code.

[junegunn]

Nice find! I never suspected folding since I myself don't usually use it.
Have you found out which LaTeX plugin is causing the issue?
Without any plugin I don't see the slowdown, even with foldenable and foldmethod set.
I wish to see it myself and possibly write a regression test case for it, before I apply your fix.
Thanks.

[junegunn]

And could you check if setlocal foldmethod=manual fixes your issue as well?

[lervag]

I'm using vim-latex, but I'm sure it also affects LaTeX-Box. I think it would happen whenever one has set foldmethod=expr for a nontrivial foldexpr function.

And yes, foldmethod=manual also fixes the problem (and might be a better solution, since foldenable opens all folds temporarily).

[junegunn]

I've tested with the following fix that temporarily overrides foldmethod to manual during alignment,
with your vim-latex plugin, and got some interesting results.

function! easy_align#align(bang, expr) range
  let st = reltime()
  let ofm = &l:foldmethod
  try
    let &l:foldmethod = 'manual'
    call s:align(a:bang, a:firstline, a:lastline, a:expr)
  catch 'exit'
  finally
    let &l:foldmethod = ofm
  endtry
  redraw
  echom reltimestr(reltime(st))
endfunction

It works great for small LaTeX files, however for very large files it can hurt performance.

The problem is, reverting &foldmethod to what it was before (e.g. let &l:foldmethod = 'expr')
can take some time for a large tex file, so the whole process becomes even slower if the table is relatively small.

Table size \ # of lines	250L	2000L	4300L	8600L
small table (9L)	0.49	0.47	0.47	0.47
small table (9L +fix)	0.49	0.61	1.16	1.96
large table (200L)	5.67	8.40	8.48	8.50
large table (200L +fix)	2.12	2.42	2.83	3.70

Files with several thousand lines are rare, but 200-line tables are also extremely uncommon.

So I'm not sure at this point if we should make this fix the default behavior of EasyAlign.
Instead, introducing a global variable, something like g:easy_align_bypass_fold, could be an option.
But then again 0.5 sec for aligning 9-line table (the one in your previous comment) doesn't seem so unbearable.
I'm still curious why it takes 10 seconds for you.

[lervag]

This is strange. The particular example where I noticed the problem is a file with about 1000L with tables of approximately 10L each. I'm happy to say that the latest versions of plugins are faster and now only take about 2 seconds for aligning, not 9. I don't know what has changed, though.

Another thing: I am not sure if your way of benchmarking is accurate (using reltime()). When I use the fix, it seems like the aligning is almost instant. Without the fix I count approximately 2 seconds. However, reltime() counts about 0.6 seconds and 1.5 seconds. I am very certain that the aligning is faster than 0.6 seconds with the fix!

If it is of help, you can look at my tex file. My reported timings come from aligning the 9 last lines from the bottommost table. I think you should also be able to notice the difference between instant (with the fix) and a delay (without the fix).

I think the fix should either be default, or at least possible to use through an option.

[junegunn]

I'm happy to say that the latest versions of plugins are faster and now only take about 2 seconds for aligning, not 9. I don't know what has changed, though.

After you created this issue, I've optimized some inefficient code, so I guess what you see is the result of it.

Another thing: I am not sure if your way of benchmarking is accurate (using reltime()).

Please make sure you don't run EasyAlign command in interactive mode when measuring the response time. The time EasyAlign command waits for keystrokes is included. Instead use non-interactive command: :EasyAlign*&

I think the fix should either be default, or at least possible to use through an option.

I think I'm going to make this feature optional with a global variable switch because it doesn't always yield the better result as I've shown you above. (FYI, neither Align.vim nor Tabular addresses this issue.)

I'll keep you updated with the progress.

[lervag]

After you created this issue, I've optimized some inefficient code, so I guess what you see is the result of it.

Nice!

Please make sure you don't run EasyAlign command in interactive mode when measuring the response time. The time EasyAlign command waits for keystrokes is included. Instead use non-interactive command: :EasyAlign*&

Ah, of course. Silly me. My timings are as follows (no fix/fix):

1. File with 8000L, 9L table: 3s / 0.04s
2. File with 8000L, 50L table: 21.3s / 0.25s
3. File with 1000L, 9L table: 0.5s / 0.06s
4. File with 1000L, 50L table: 3.2s / 0.24s
5. File with 250L, 9L table: 0.14s / 0.06s
6. File with 250L, 50L table: 0.8s / 0.25s

As you see, I do not experience the lag due to changing the foldmethod that you reported, and with the fix, the timings are comparable for all file sizes.

I think I'm going to make this feature optional with a global variable switch because it doesn't always yield the better result as I've shown you above. (FYI, neither Align.vim nor Tabular addresses this issue.)

I'll keep you updated with the progress.

I understand that you want to keep this change optional as long as you are not fully convinced that it is always optimal. And I am very happy that you want to address this at all :)

[junegunn]

It seems that the cost of setting &foldmethod to expr depends not only on the number of
lines in the file, but also on the complexity of the file structure.

let st = reltime() | let &l:foldmethod='expr' | redraw | echom reltimestr(reltime(st))

The above code takes 0.42 seconds for mostly-empty 8000L LaTeX file,
but 1.65 seconds for 8000L file with rather complex structure with many nested folds.

And of course, the fold cost will vary from plugin to plugin.
So it's really not easy to predict how long it will take.

[lervag]

I just now noticed that you add redraw. I see no reason why you want to redraw. At least everything works as I expect without redraw, both in vim and gvim. Adding redraw easily explains your timings, but I really don't think it should be necessary.

[junegunn]

If we do it without redraw,

let st = reltime() | let &l:foldmethod='expr' | echom reltimestr(reltime(st))

it returns faster, echom prints short response time, but I noticed that Vim is unresponsive after the message for some period, and it seemed to me that adding redraw is required to measure the actual delay a user experiences.

[junegunn]

There is one thing I just realized:

let st = reltime() | setlocal nofoldenable | setlocal foldenable | redraw | echom reltimestr(reltime(st))

This is almost free compared to setting foldmethod!

[junegunn]

Ah, nevermind. setlocal nofoldenable alone doesn't seem to cut down the time for alignment

[lervag]

Hmm, perhaps you are right. At least I do not experience any difference in responsiveness if I add redraw.

I feel like there should be some way of changing lines that does not invoke the folding. I tried a quick search, but unfortunately I don't find anything about this.

[junegunn]

https://github.com/junegunn/vim-easy-align#disabling-foldmethod-during-alignment

Updated with g:easy_align_bypass_fold. Could you confirm that it works as expected?

[lervag]

It seems to work well, thanks!

[junegunn]

Good to hear that. I'll close this issue for the time being. If you happen to find a better approach to tackle this problem, feel free to let me know. Thanks.