Use Apache HTTP client

[benjamingeer]

This PR changes HttpTriplestoreConnector, SipiResponderV1, and SipiResponderV2 to use Apache HttpClient instead of the Akka HTTP client API.

The aim of this PR is to reduce the amount of time we have to spend on HTTP client issues. We've had a series of time-consuming issues involving difficult-to-explain Akka HTTP exceptions, including:

• akka.http.impl.engine.client.OutgoingConnectionBlueprint$UnexpectedConnectionClosureException: Load testing #741
• java.lang.IllegalStateException: Substream Source cannot be materialized more than once: Triplestore connection error when using dockerComposeUp #1122

There's also the problem of freezing on macOS: akka/akka#25605.

Akka HTTP is a relatively new project, while Apache HttpClient is nearly 20 years old and still actively developed. Its API also seems simpler and easier to understand. And I think we don't use or need most of the features of the Akka HTTP client API.

Fixes #1122.
Fixes #1081.

[benjamingeer]

@gfoo Could you please try this branch and see if it fixes #1081 for you?

[gfoo]

yes, it works. Notice that I use: gdbImage := "dhlabbasel/graphdb-free:8.3.1".

[subotic]

I don't think that GraphDB 8.3.1 is supported anymore. There are some queries, that require GraphDB 8.5.0.

[subotic]

I've sent you all a license. GraphDB-Free is resource intensive and slow.

[subotic]

We tried it on Travis, and could never get it to work. Always had some timeouts. If you have a fast CPU and not a lot of data, then it will maybe work.

[gfoo]

Nom du modèle :	MacBook Pro
  Identifiant du modèle :	MacBookPro11,4
  Nom du processeur :	Intel Core i7
  Vitesse du processeur :	2,5 GHz
  Nombre de processeurs :	1
  Nombre total de cœurs :	4
  Cache de niveau 2 (par cœur) :	256 Ko
  Cache de niveau 3 :	6 Mo
  Mémoire :	16 Go

Conf Docker:

CPU 6
Mem 8 Go

GraphDB-Free is resource intensive and slow.

ah ok, maybe it explains my noisy fan so often!

Anyway I now use your license, thanks for that.

[benjamingeer]

@subotic Which GraphDB SE version should we be using now?

[subotic]

@subotic Which GraphDB SE version should we be using now?

8.5.0: https://github.com/dhlab-basel/Knora/blob/develop/project/Dependencies.scala

If you run docker-compose from SBT, then this is already set correctly.

There are some memory issues with later versions, which we are still working on.

[subotic]

LGTM. We should run the load-tests to be sure. I'm on vacation from tomorrow on, so I could do it in January, at the earliest. You just need to run run.py from here: https://github.com/dhlab-basel/knora-tests/tree/master/import-test

[benjamingeer]

We should run the load-tests to be sure.

Thanks, I‘ll do it tomorrow.

[subotic]

Just to have it on record. I find this at most a temporary fix. The problem is not Akka HTTP, it is more likely that the way we are using Akka and Akka HTTP is producing the problem. Just exchanging one technology with another because we don't understand it, is not really the solution. Now we have Akka, Akka HTTP, and Apache HTTP. Our library dependency tree has yet again considerably. We are going to pay the price at some other point.

Next year we should try to find a solution to the real problem.

[benjamingeer]

The problem is not Akka HTTP

Perhaps. But all software has bugs. And newer software with a very innovative design is perhaps likely to have more bugs than older software with a more conventional design. When we first tried Akka HTTP, it had a memory leak, which I reported to them, and they confirmed and fixed it. And they thanked me for the bug report: akka/akka#16900 (comment)

It actually seems to me that we use very few features of Akka in general, and perhaps we don't really need it at all. But let's talk about this next year.

[benjamingeer]

The load test completed without errors.