Failed to start local backend enviroment

[RobertIndie]

Body

Hi, all

The core developement doc seems wrong because we have moved the streampipes-backend to streampipes-core in this commit.

Therefore I try to use this configuration: https://github.com/apache/streampipes/blob/dev/.idea/runConfigurations/core.xml to start the local backend. But it raises these errors:

2023-02-21T00:16:03.250+08:00  INFO 63389 --- [pool-1-thread-1] o.a.s.s.c.StreamPipesBackendApplication  : No more pipelines to restore...
2023-02-21T00:17:03.625+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.7:8001: Connection reset
2023-02-21T00:17:03.627+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.7:8001
2023-02-21T00:17:08.649+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.7:8001: Connection reset
2023-02-21T00:17:08.649+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.7:8001
2023-02-21T00:17:13.651+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.7:8001: Connection reset
2023-02-21T00:17:13.651+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.7:8001
2023-02-21T00:17:18.681+08:00  INFO 63389 --- [pool-3-thread-1] o.a.s.m.m.p.ExtensionsServiceLogExecutor : Could not fetch log info from endpoint http://172.31.0.7:8001
2023-02-21T00:17:23.831+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.5:8090: Connection reset
2023-02-21T00:17:23.836+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.5:8090
2023-02-21T00:17:28.857+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.5:8090: Connection reset
2023-02-21T00:17:28.859+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.5:8090
2023-02-21T00:17:33.917+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.5:8090: Connection reset
2023-02-21T00:17:33.918+08:00  INFO 63389 --- [pool-3-thread-1] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.5:8090

And the UI container also cannot connect to the backend due to:

2023-02-21 00:19:01 172.31.0.1 - - [20/Feb/2023:16:19:01 +0000] "GET /streampipes-backend/api/v2/setup/configured HTTP/1.1" 502 559 "http://localhost/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
2023-02-21 00:19:02 2023/02/20 16:19:02 [error] 7#7: *2 backend could not be resolved (3: Host not found), client: 172.31.0.1, server: , request: "GET /streampipes-backend/api/v2/setup/configured HTTP/1.1", host: "localhost", referrer: "http://localhost/"

Not sure why. There may be some problems in the environment setting.
Do you have the same issue? Could you provide a new doc to develop the backend? Thanks!

StreamPipes Committer

I acknowledge that I am a maintainer/committer of the Apache StreamPipes project.

[dominikriemer]

Hi @RobertIndie thanks for reporting this! I'll update the documentation.

I haven't seen this error so far, but I checked my startup configuration and it doesn't differ from the xml.
So I guess this maybe might be an issue related to backwards compatibility.
Are there any other errors in your console when starting the backend?
Do you see any error messages related to Consul or CouchDB?

From a very first glance it seems the UI tries to connect to the backend host (which is probably unreachable when you're running the backend locally) - are you starting the UI in Docker and the backend locally or which setup are you running?

[dominikriemer]

The backend host I'm referring to is defined in the nginx configuration in the ui/nginx_config/default.conf file.

[RobertIndie]

From a very first glance it seems the UI tries to connect to the backend host (which is probably unreachable when you're running the backend locally) - are you starting the UI in Docker and the backend locally or which setup are you running?

I'm starting it using commands ./streampipes env --set backend and ./streampipes up -d. The UI is started in the docker:

streampipes/installer/cli/environments/backend

Line 23 in 0f6bcf5

ui

Shouldn't it be started in the docker?

Are there any other errors in your console when starting the backend?
Do you see any error messages related to Consul or CouchDB?

I see that the backend trying to connect to port 8090, but there is not service exposed the port 8090 in the docker.
Here is the log I found from the backend:

2023-02-21T20:02:24.827+08:00  INFO 8324 --- [           main] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.6:8090: Connection reset
2023-02-21T20:02:24.828+08:00  INFO 8324 --- [           main] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.6:8090
2023-02-21T20:02:29.830+08:00  INFO 8324 --- [           main] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.6:8090: Connection reset
2023-02-21T20:02:29.830+08:00  INFO 8324 --- [           main] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.6:8090
2023-02-21T20:02:34.831+08:00  INFO 8324 --- [           main] o.apache.http.impl.execchain.RetryExec   : I/O exception (java.net.SocketException) caught when processing request to {}->http://172.31.0.6:8090: Connection reset
2023-02-21T20:02:34.832+08:00  INFO 8324 --- [           main] o.apache.http.impl.execchain.RetryExec   : Retrying request to {}->http://172.31.0.6:8090
2023-02-21T20:02:39.837+08:00  WARN 8324 --- [           main] o.a.s.m.endpoint.EndpointItemFetcher     : Processing Element Descriptions could not be fetched from RDF endpoint: http://172.31.0.6:8090
2023-02-21T20:02:39.840+08:00  INFO 8324 --- [           main] o.a.s.manager.setup.InstallationStep     : Installing pipeline elements from http://172.31.0.6:8090