Frequent 'Unable to initialize IDE' error on AWS

[kaloyan-raev]

I have an issue running Che on an AWS EC2 instance. The browser fails to initialize the Web IDE showing a popup dialog saying 'Unable to initialize IDE'.

The workspace seems to be initialized correctly. I can verify this using the Che Terminal panel, which is not affected by the error.

Unfortunately, the Project Explorer and Commands drop-down list are empty and the IDE is quite unusable.

I noticed that the issue is quite frequent, but not 100% reproducible. It's more like 80% reproducible. It happens only with stacks that have Language Server agents enabled by default (like the PHP stack).

I found the following error in the Chrome Console:

ERROR (org.eclipse.che.plugin.languageserver.ide.LanguageServerFileTypeRegister): 
java.lang.Exception: 
	at Unknown.nm(_app-0.js)
	at Unknown.vm(_app-0.js)
	at Unknown.Qfj(_app-0.js)
	at Unknown.Sfj(_app-0.js)
	at Unknown.R_d(_app-0.js)
	at Unknown.eval(_app-0.js)

The issue is not reproducible with an older installation of Che - version 5.0.0-M8.

Reproduction Steps:

1. I have a m4.large instance running on AWS EC2.
2. All inbound and outbound traffic is allowed in the attached security group.
3. Che is started with the following commands:

export CHE_IP=`curl -s http://169.254.169.254/latest/meta-data/public-ipv4`
docker run -it --rm -v /var/run/docker.sock:/var/run/docker.sock -v /opt/che-data:/data -e CHE_DOCKER_IP_EXTERNAL=$CHE_IP eclipse/che start

4. Open the Dashboard in the browser.
5. Start a workspace with the PHP stack.

The workspace will initialize correctly, but the browser will fail initializing the Web IDE.

OS and version:

Ubuntu 14.04.4 LTS (running on AWS EC2)

Diagnostics:

WARN: Bound 'eclipse/che' to 'eclipse/che:5.3.1'
WARNING: No swap limit support
INFO: (che cli): 5.3.1 - using docker 1.12.3 / native

CLI:
 TTY:            true
 Daemon:         /var/run/docker.sock
 Image:          eclipse/che
 Version:        5.3.1
 Command:        info
 Parameters:     info
Mounts:
 /data:          /opt/che-data
 /data/instance: not set
 /data/backup:   not set
 /repo:          not set
 /assembly:      not set
 /sync:          not set
 /unison:        not set
 /chedir:        not set
System:
 Docker:         native
 Proxy:          not set
Internal:
 CHE_VERSION:    5.3.1
 CHE_HOST:       172.30.2.125
 CHE_INSTANCE:   /opt/che-data/instance
 CHE_CONFIG:     /opt/che-data
 CHE_BACKUP:     /opt/che-data/backup
 CHE_REGISTRY:   /version
 CHE_DEBUG:      false
 IP Detection:   172.30.2.125
 Initialized:    true
Image Registry:
 IMAGE_INIT=eclipse/che-init:5.3.1
 IMAGE_CHE=eclipse/che-server:5.3.1
 IMAGE_COMPOSE=docker/compose:1.8.1
 BOOTSTRAP_IMAGE_ALPINE=alpine:3.4
 BOOTSTRAP_IMAGE_CHEIP=eclipse/che-ip:5.3.1
 UTILITY_IMAGE_CHEACTION=eclipse/che-action:5.3.1
 UTILITY_IMAGE_CHEDIR=eclipse/che-dir:5.3.1
 UTILITY_IMAGE_CHETEST=eclipse/che-test:5.3.1
 UTILITY_IMAGE_CHEMOUNT=eclipse/che-mount:5.3.1
che.env:
 CHE_HOST=172.30.2.125

[bartlomiej-laczkowski]

The same situation here, I also get the same popup when starting workspace created from i.e. PHP stack.

[bartlomiej-laczkowski]

I am using local build of Che 5.4.0, below is some additional info about my environment:

OS and version:

Windows 10 + Docker 1.13.1

Diagnostics:

CLI:
 TTY:            true
 Daemon:         /var/run/docker.sock
 Image:          eclipse/che:nightly
 Version:        nightly
 Command:        info
 Parameters:     info
Mounts:
 /data:          /c/tmp/che
 /data/instance: not set
 /data/backup:   not set
 /repo:          /c/Users/Bartek/git/che-bart
 /assembly:      not set
 /sync:          not set
 /unison:        not set
 /chedir:        not set
System:
 Docker:         docker4windows
 Proxy:          not set
Internal:
 CHE_VERSION:    nightly
 CHE_HOST:       10.0.75.2
 CHE_INSTANCE:   /c/tmp/che/instance
 CHE_CONFIG:     /c/tmp/che
 CHE_BACKUP:     /c/tmp/che/backup
 CHE_REGISTRY:   /version
 CHE_DEBUG:      false
 IP Detection:   10.0.75.2
 Initialized:    true
 CHE_DEVELOPMENT_REPO: /c/Users/Bartek/git/che-bart
Image Registry:
 IMAGE_INIT=eclipse/che-init:nightly
 IMAGE_CHE=eclipse/che-server:nightly
 IMAGE_COMPOSE=docker/compose:1.8.1
 BOOTSTRAP_IMAGE_ALPINE=alpine:3.4
 BOOTSTRAP_IMAGE_CHEIP=eclipse/che-ip:nightly
 UTILITY_IMAGE_CHEACTION=eclipse/che-action:nightly
 UTILITY_IMAGE_CHEDIR=eclipse/che-dir:nightly
 UTILITY_IMAGE_CHETEST=eclipse/che-test:nightly
 UTILITY_IMAGE_CHEMOUNT=eclipse/che-mount:nightly
che.env:
 CHE_HOST=10.0.75.2

[ghost]

@kaloyan-raev is this a public instance that I can access?

@bartlomiej-laczkowski I will give it a try with the latest tag on Win 10

[kaloyan-raev]

It's much harder to reproduce in Super Dev Mode. But I was able to once see a more detailed exception in the Chrome debugger:

Error: Editor initializer not set
    at StackTraceCreator$CollectorModern.collect_0 [as collect] (_app-0.js:12085:28)
    at Object.captureStackTrace (_app-0.js:11916:22)
    at RuntimeException_0.fillInStackTrace (_app-0.js:10133:10)
    at RuntimeException_0.Throwable_0 (_app-0.js:10080:8)
    at RuntimeException_0.Exception_0 (_app-0.js:10208:18)
    at RuntimeException_0 (_app-0.js:10248:18)
    at EditorInitializePromiseHolder.getInitializerPromise (_app-0.js:180228:11)
    at OrionContentTypeRegistrant.registerFileType [as registerFileType_0] (_app-0.js:180495:37)
    at LanguageServerFileTypeRegister$1.apply_1083 [as apply_29] (_app-0.js:380167:44)
    at LanguageServerFileTypeRegister$1.apply_1082 [as apply_6] (_app-0.js:380133:8)
    at _app-0.js:102002:17"

It seems that the OrionEditorExtention is not yet initialized at the time the LS FileTypes are registered in the Orion editor.

[kaloyan-raev]

@eivantsov Yes. Try this URL: http://107.22.151.190:8080/dashboard/#/ide/che/wksp-qcf5

If it does not happen the first time, try refreshing the browser until you see it.

[ghost]

@kaloyan-raev do you think this is affected by some latency? i.e. the fact that this is a remote instance?

[kaloyan-raev]

It seems there is some race condition. I haven't seen this locally on my Linux system. @bartlomiej-laczkowski sees it often on his Windows, but there Docker is installed differently than on Linux. And on this AWS instance is quite often. But never happened with an older 5.0.0-M8 installation...

[ghost]

Yes, Win 10 is kind of a remote Che as well since containers run in a VM.

[kaloyan-raev]

Perhaps some async calls from browser to Che server take longer time than assumed in these remote use cases.

[ghost]

Yes, I can see the same behavior.

I'll start a remote Che and the IDE team will take a look.

[kaloyan-raev]

Is there a way to ensure that the OrionEditorExtension constructor is called before the LanguageServerFileTypeRegister.start() method?

[kaloyan-raev]

Just pushed PR #4352 with a fix.

[dmytro-ndp]

I have also reproduced this issue with PHP workspace which is obtained from stack on nighlty version of Che locally:

[vparfonov]

cannot reproduce it

[ciastirfry]

I have the same issue here. Is there a step by step solution for this?

[ghost]

@ciastirfry can you elaborate? Does it happen to any workspace that you start? Please note that you need to open the ephemeral port range on your AWS VM otherwise your browser would never reach the workspace agent which is a Tomcat server running in a Docker container on port 8080 which is exposed to a random port from the ephemeral port range each time the workspace starts.

[ciastirfry]

@eivantsov, thanks for your immediate response. are you referring to the port below. Any steps how to open that in AWS? I'm using Ec2.

32768-65535 | Docker and Che Agents

[ghost]

@ciastirfry yes, I am referring to the ephemeral port range 32768-65535. Perhaps docs on security groups can help http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html

[ciastirfry]

@eivantsov, We'll try this approach. Thanks a lot for your help!

[divgo]

For anyone else that has this same issue, I found a different fix for it.
I was receiving the error when accessing Che from a remote machine. I have Che running on my Mac, but when I would access it from my Chromebook, the Workspace would fail to load with the noted error.

The fix for me was to stop the container and edit che.env
roughly around line 206:

# Che Server API Endpoint
#     The location of the API end point where dashboard and IDE clients will look for
#     interacting with the Che server, which we also call the workspace master.
CHE_API=http://sage-imac:${SERVER_PORT}/wsmaster/api

Once I changed that and started the container, the Workspaces would load immediately on my Chromebook.

That setting by default is pointing to http://localhost:${SERVER_PORT}