Add DA and ptvsd experiments support to remote debugging API

[kimadeline]

For #7549

• Pull request represents a single change (i.e. not fixing disparate/unrelated things in a single PR)
• Title summarizes what is changing
• Has a news entry file (remember to thank yourself!)
• Appropriate comments and documentation strings in the code
• Has sufficient logging.
• Has telemetry for enhancements.
• Unit tests & system/integration tests are added/updated
• Test plan is updated as appropriate
• package-lock.json has been regenerated by running npm install (if dependencies have changed)
• The wiki is updated with any design decisions/details.

[codecov-io]

Codecov Report

Merging #7570 into master will decrease coverage by 0.65%.
The diff coverage is 77.77%.

@@            Coverage Diff             @@
##           master    #7570      +/-   ##
==========================================
- Coverage   58.88%   58.23%   -0.66%     
==========================================
  Files         492      492              
  Lines       21851    21521     -330     
  Branches     3503     3505       +2     
==========================================
- Hits        12868    12533     -335     
- Misses       8182     8186       +4     
- Partials      801      802       +1

Impacted Files	Coverage Δ
src/client/debugger/extension/types.ts	100% <ø> (ø)	⬆️
src/client/api.ts	90% <100%> (+2.5%)	⬆️
src/client/debugger/extension/adapter/factory.ts	91.66% <50%> (-2.62%)	⬇️
...nt/datascience/jupyter/invalidNotebookFileError.ts	0% <0%> (-75%)	⬇️
.../client/datascience/jupyter/jupyterConnectError.ts	0% <0%> (-66.67%)	⬇️
...science/jupyter/jupyterKernelPromiseFailedError.ts	0% <0%> (-50%)	⬇️
...ent/datascience/jupyter/jupyterWaitForIdleError.ts	0% <0%> (-50%)	⬇️
src/client/language/characters.ts	12% <0%> (-21.34%)	⬇️
src/client/testing/codeLenses/main.ts	60% <0%> (-17.78%)	⬇️
src/client/datascience/cellFactory.ts	72.34% <0%> (-17.32%)	⬇️
... and 36 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 51cdc7e...98a3de5. Read the comment docs.

[ericsnowcurrently]

Mostly LGTM. I've left some formatting-related comments (appropriately marked) that you can do with as you will. I also provided some points to consider regarding getRemoteLauncherCommand(). FWIW, the logic seems fine. The only real concern I have is that DebugAdapterDescriptorFactory.getPtvsdPath() is returning the wrong thing.

[ericsnowcurrently]

From what I understand, the await debugFactory.useNewPtvsd(pythonSettings.pythonPath)) case is for when a user is using their own ptvsd. If that's the case then won't that case still apply after the experiment is over? We would be stuck still depending on the old debug adapter code (e.g. RemoteDebuggerExternalLauncherScriptProvider).

So this is probably the best opportunity to address that. Do that by folding RemoteDebuggerExternalLauncherScriptProvider into getRemoteLauncherCommand() here or into DebugAdapterDescriptorFactory.getPtvsdPath().

I suppose we could also punt on it (open a new issue). :)

[ericsnowcurrently]

Anyway, the key question is whether or not the user gets a version of ptvsd that has the new debug adapter. Really the key question is which "script" to use. So it may make more sense to have DebugAdapterDescriptorFactory.getPtvsdPath() deal with the cases. Then DebugAdapterDescriptorFactory.useNewPtvsd() would be unnecessary. That would mean getRemoteLauncherCommand() would look more like this:

            async getRemoteLauncherCommand(host: string, port: number, waitUntilDebuggerAttaches: boolean = true): Promise<string[]> {
                const script = await debugFactory.getPtvsdPath(pythonSettings.pythonPath);
                const waitArgs = waitUntilDebuggerAttaches ? ['--wait'] : [];
                return [
                    script.fileToCommandArgument(),
                    '--default',
                    '--host', host,
                    '--port', port.toString(),
                    ...waitArgs
                ];
            }

DebugAdapterDescriptorFactory.getPtvsdPath() would have to deal with the 3 cases (new debug adapter, old debug adapter with our distributed ptvsd, user-installed ptvsd).

FWIW, I'm also not a fan of how ptvsd-specific details are sitting in this module. They should be with the other debugger stuff. The above helps address that a little. I'd expect a better solution to involve adding something like DebugAdapterDescriptorFactory.getPTVSDArgv(). We can look into addressing this separately. :)

[ericsnowcurrently]

LGTM