Variable Explorer perf issue

[ronglums]

from survey:

When working in interactive Python, the variable refresh is quite slow. After every batch send to IPython it refreshes.

[rchiodo]

This should be fixed already. Do you have any more information? @IanMatthewHuff fixed this in the last release I believe.

[rchiodo]

Yeah this was fixed in microsoft/vscode-python#5708 which was shipped with our latest release.

[greazer]

We're assuming that this is fixed. If not, please enter a new issue.

[rchiodo]

Here's some code that repros the problem:

# %%
i = 0

#%%
%%time
exec("var{}=[{} ** 10 % 17 for l in range(100000)]".format(i, i))
i = i + 1

Execute the first cell.
Execute the second cell like 100 times.

Open the variable explorer
Execute the second cell.

It will take 4 or 5 seconds for the cell to finish executing.

[rchiodo]

This is related to https://github.com/microsoft/vscode-python/issues/7180

[rchiodo]

3 different ideas:

1. Modify the debugger to support an eval that gets us this list and allows us to return data in chunks
2. Modify our python code to return all of the data needed for one screen worth
3. Modify our python code to return just the list of variables, use JMP 'inspect_request' to return data shown above.

[rchiodo]

Pros/Cons of each

Idea	Pros	Cons
Debugger	Outside execution path (async)	Python specific
	Already does this sort of thing	Can be slow too
		Remote not working yet
Python Code	Quickest to get working	Still python only
		Execution based (not async)
Jupyter Hybrid	Should be portable to other languages
	Might be async with execution?

[rchiodo]

Here's an even better test:

for i in range(1000):
    exec("var{}=[{} ** 10 % 17 for l in range(100000)]".format(i, i))

Creates 1000 lists with 10000 items in each.

[IanMatthewHuff]

There was a bit of a regression that I filed for #2382, but overall looks good. Validated.