extensions.all API does not return all installed extensions

[Eskibear]

Does this issue occur when all extensions are disabled?: Yes/No

• VS Code Version: 1.71.2
• OS Version: Linux/Windows

Context

Java extension allows other extensions to contribute server bundles (plugin of Java Language Server). The client side calls vscode.extensions.all to get all enabled extensions and searches for javaExtensions definition in packageJSON, and sends to the server.

Spring extension depends on Java extension, and provides a server bundle for spring-specific features.

Problem

Original issue: spring-projects/sts4#848

While activating vscode-java extension, vscode.extension.all did return a full list of installed extensions, which causes Java LS didn't load all required bundles, resulting to unexpected errors.

Steps to Reproduce:

I am using this simple project which contains a devcontainer.json to have all the necessary extensions on board: https://github.com/martinlippert/codespaces-spring-demo - in case you want to give this a try

Sometimes it works, sometimes it doesn't. I used Rebuild container in order to trigger the Codespace creation from the beginning in case it doesn't work

by @martinlippert

Logs

verbose intialize request sent to server, you see a list of bundles but those provided by the Spring extension are not included in this run.
Ref: how vscode-java extension collect the bundles

"initializationOptions": {
        "bundles": [
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-debug-0.44.0/server/com.microsoft.java.debug.plugin-0.40.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-dependency-0.21.0/server/com.microsoft.jdtls.ext.core-0.21.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-jupiter-api_5.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-jupiter-engine_5.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-jupiter-migrationsupport_5.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-jupiter-params_5.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-commons_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-engine_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-launcher_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-runner_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-suite-api_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-suite-commons_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-platform-suite-engine_1.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/junit-vintage-engine_5.9.0.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/org.apiguardian.api_1.1.2.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/org.eclipse.jdt.junit4.runtime_1.3.0.v20220609-1843.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/org.eclipse.jdt.junit5.runtime_1.1.0.v20220715-1030.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/org.opentest4j_1.2.0.v20190826-0900.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-java-test-0.37.1/server/com.microsoft.java.test.plugin-0.37.1.jar",
            "/home/vscode/.vscode-remote/extensions/vscjava.vscode-maven-0.38.0/jdtls.ext/com.microsoft.java.maven.plugin/target/com.microsoft.java.maven.plugin-0.37.0.jar"
        ],

Log (Remote Extension Host)

[2022-09-21 14:11:04.606] [exthost] [info] ExtensionService#_doActivateExtension vscjava.vscode-java-dependency, startup: false, activationEvent: 'onLanguage:java'
[2022-09-21 14:11:04.694] [exthost] [info] ExtensionService#_doActivateExtension vscjava.vscode-java-debug, startup: false, activationEvent: 'onLanguage:java'
[2022-09-21 14:11:04.762] [exthost] [info] ExtensionService#_doActivateExtension redhat.java, startup: false, activationEvent: 'onLanguage:java'
[2022-09-21 14:11:04.894] [exthost] [info] ExtensionService#_doActivateExtension vscjava.vscode-java-pack, startup: false, activationEvent: 'onLanguage:java'
[2022-09-21 14:11:05.374] [exthost] [info] ExtensionService#_doActivateExtension vscjava.vscode-maven, startup: false, activationEvent: 'workspaceContains:**/pom.xml'
[2022-09-21 14:11:05.601] [exthost] [warning] [redhat.java] Accessing a resource scoped configuration without providing a resource is not expected. To get the effective value for '[java]', provide the URI of a resource or 'null' for any resource.
[2022-09-21 14:11:17.311] [exthost] [info] ExtensionService#_doActivateExtension vscjava.vscode-java-test, startup: false, activationEvent: 'onLanguage:java'
[2022-09-21 14:11:17.472] [exthost] [info] ExtensionService#_doActivateExtension Pivotal.vscode-spring-boot, startup: false, activationEvent: 'onLanguage:java'
[2022-09-21 14:11:23.219] [exthost] [error] Error: No delegateCommandHandler for sts.java.addClasspathListener
    at /home/vscode/.vscode-remote/extensions/redhat.java-1.10.0-linux-x64/dist/extension.js:2:1034658
    at /home/vscode/.vscode-remote/extensions/redhat.java-1.10.0-linux-x64/dist/extension.js:2:1034952
    at Immediate.<anonymous> (/home/vscode/.vscode-remote/extensions/redhat.java-1.10.0-linux-x64/dist/extension.js:2:1035317)
    at processImmediate (node:internal/timers:466:21) java.execute.workspaceCommand {"value":"redhat.java","_lower":"redhat.java"}

Implementation of command sts.java.addClasspathListener is in those provided by the Spring extension. This error is because they are not loaded.

[Eskibear]

We also reproduce it on desktop version vscode sometimes while testing Java extension pack (containing Java extension and other extensions depending on it), only in less-powerful machines. And that's why I decide to open it here instead of vscode-remote.

Below are some notes taken by testing team when the error occurred... And from logs we also find that client didn't include corresponding jars in initializationOptions.bundles, same as above.

1st occurrence:

• Install Extension Pack for Java v0.25.1
• Clone and open https://github.com/codecentric/spring-boot-admin.git
• After import completed, open Java Project window > Refresh > Show an error （Debug, also show the error）
  

2nd occurrence:

• repo: https://github.com/sephiroth74/android-target-tooltip.git
• Clone and open the project
• Wait for import completed
• Open a java file, F5 to debug
• The "no delegate command" error occurs
• Reload window and it gets back to work.

Note, before both occurrences, the user removed some cache folders from %appdata%. Not sure if that's related.

/cc @jdneo @testforstephen

[Eskibear]

Update:
We did some experiments and got more findings. Hope it helps you understand the use case.

At the time Java extension calls extensions.all, some extensions are not "currently known to the system" . Java LS starts without loading bundles from those extensions. Java extension does listen to the changes of extensions and prompt a notification for that. But currently the listener is registered after LS is ready. In above cases where the issue occurs, the extensions.didChange event was fired while LS was launching, after client sent the initialize request and before the listener got registered.

There is some sort of "race condition" between loading all enabled extensions into ExtensionDescriptionRegistry and java extension calling the API. That explains why we sometimes see the issue on less-performant machines.

Follow-up actions
For Java extension: /cc @jdneo @testforstephen @rgrunber

• double check extensions.all after LS is ready before listening to extension changes, check whether that's changed during that period of time.
• explore possibility of dynamically loading bundles into Java LS.

[jrieken]

Adding @alexdima for the details of extension loading

double check extensions.all after LS is ready before listening to extension changes

That sounds like the correct thing to do anyways

Also note that there is #145307 to discover extensions from different extension host (remote, local, web)

[alexdima]

Reading the original issue:

I observed this when working with the remote development extension in VSCode and Codespaces as well as local dev containers running on Docker. It doesn't happen always, but on a recurring base and definitely more often than when running VSCode locally with a local workspace. I have no idea what is causing this though.

When creating a new Codespace or a new dev container, the VSCode UI (web or desktop) will come up before the remote extensions have finished installing. The extensions will install in parallel with the UI loading. To give a couple of examples when this would happen (cc @sandy081 to double check what I'm describing is correct):

• if a user has a certain extension installed, and the user uses settings sync, when the Codespace comes up the extension will be installed via settings sync. But this will not block the Codespace coming up.
• similarly, if the project uses a devcontainer.json file and lists certain extensions to install, they will be installed after VS Code comes up.
  This is intentional behavior, as we had to optimize towards getting the UI up and running as soon as possible. This behavior can also be emulated locally by installing an extension after VS Code starts up.

But currently the listener is registered after LS is ready. In above cases where the issue occurs, the extensions.didChange event was fired while LS was launching, after client sent the initialize request and before the listener got registered.

I think this might be the root problem. Perhaps you are reading extensions.all, do a bunch of async stuff which might take a long time on slow machines, during this time some extensions get installed, the event is fired by us, but then you register the event listener only later.

Could you please change on your side to register the extensions change event at the point when you read extensions.all since that is the most likely explanation?

[Eskibear]

@jrieken @alexdima
Thank you both for the detailed explanation, which is very helpful for us to understand the mechanism. The behavior of extensions.all makes sense to me now. Feel free to close the issue then.

BTW, @jdneo is changing the implementation accordingly on our side.