Update to MongoDB Java driver 4.0.0

[Postremus]

We definitely want a note about this change in the migration guides.
The interface changes for reactive mongo most likely break user applications.

See also the mongo java driver 4.0 upgrade guide: https://mongodb.github.io/mongo-java-driver/4.0/upgrading

[loicmathieu]

I will try this one on some projects I have that uses more Mongodb functionalities before approving this. I want to be sure that everything works as before with this new major version of the client

[cescoffier]

I've added one comment about the new "TDocument" thingy.

Also, do you know why https://docs.mongodb.com/ecosystem/drivers/java/ has not been updated with this version? I would like to be sure we don't lose support for some MongoDB version. If we do we must document it.

[cescoffier]

I'm not sure if this should be considered as a breaking change.
Initially, it only contained Document, now it can be any type.

First I would use TDocument as in the driver to indicate that it's not any type, but must be somehow a document. The lack of constraint in the driver is a bit odd.

[loicmathieu]

@cescoffier TDocument means any type T that can be converted to a document. I agree this is a bit odd as it didn't convey any constraint. As we register the Pojo Codec, almost any type can be converted automatically (except for some deep generic types). So maybe T is OK.

[Postremus]

@cescoffier

No Idea why the table on the official site was not updated.
In https://mongodb.github.io/mongo-java-driver/4.0/upgrading/#compatibility, It has the new 4.0 driver, with same compatibility as our old 3.12.0.

[loicmathieu]

I check some internal projects (including some basic Kotlin ones) and all works without issue.

It is worth noting that the uber client is no longer available, but as we switch to the new clients in the previous release this should not be an issue.

It is also worth noting that quarkus-camel also provides MongoDB support, maybe we should check on their side before merging this one, @gsmet ?

[gastaldi]

It is also worth noting that quarkus-camel also provides MongoDB support, maybe we should check on their side before merging this one, @gsmet ?

Maybe @ppalaga or @lburgazzoli can confirm?

[ppalaga]

It is also worth noting that quarkus-camel also provides MongoDB support, maybe we should check on their side before merging this one, @gsmet ?

Maybe @ppalaga or @lburgazzoli can confirm?

@jamesnetherton is our Mongo expert

[jamesnetherton]

I ran the camel-quarkus integration tests with this change...

Unfortunately, the camel-mongo component is currently tied to the 3.x drivers and there are API changes in 4.x which seem to break things. E.g:

Caused by: java.lang.NoSuchMethodError: com.mongodb.client.MongoCollection.insertOne(Ljava/lang/Object;)V
	at org.apache.camel.component.mongodb.MongoDbProducer.lambda$createDoInsert$7(MongoDbProducer.java:437)
	at org.apache.camel.component.mongodb.MongoDbProducer.lambda$wrap$0(MongoDbProducer.java:256)
	at org.apache.camel.component.mongodb.MongoDbProducer.invokeOperation(MongoDbProducer.java:136)
	at org.apache.camel.component.mongodb.MongoDbProducer.process(MongoDbProducer.java:123)

What we probably want, is a camel-mongo4 component which uses the newer Mongo libs and APIs. I can't say how much effort that'd be or when we're likely to have that though.

[loicmathieu]

@jamesnetherton it's surprising because we do use insertOne inside MongoDB with Panache:

quarkus/extensions/panache/mongodb-panache/runtime/src/main/java/io/quarkus/mongodb/panache/runtime/MongoOperations.java

Line 195 in 36650ac

collection.insertOne(entity);

Did you check that you don't have a dependency issue and that all mongodb related libraries are of version 4 ?

[jamesnetherton]

@loicmathieu you're right, I think it should work.

Dependencies look fine:

[INFO] --- maven-dependency-plugin:3.1.1:tree (default-cli) @ camel-quarkus-integration-test-mongodb ---
[INFO] org.apache.camel.quarkus:camel-quarkus-integration-test-mongodb:jar:1.1.0-SNAPSHOT
[INFO] \- org.apache.camel.quarkus:camel-quarkus-mongodb:jar:1.1.0-SNAPSHOT:compile
[INFO]    +- io.quarkus:quarkus-mongodb-client:jar:999-SNAPSHOT:compile
[INFO]    |  +- org.mongodb:mongodb-driver-sync:jar:4.0.0:compile
[INFO]    |  \- org.mongodb:mongodb-driver-reactivestreams:jar:4.0.0:compile
[INFO]    +- org.mongodb:mongodb-driver-legacy:jar:4.0.0:compile
[INFO]    |  +- org.mongodb:bson:jar:4.0.0:compile
[INFO]    |  \- org.mongodb:mongodb-driver-core:jar:4.0.0:compile
[INFO]    \- org.mongodb:mongodb-crypt:jar:1.0.0:compile

Here's the failure point:

https://github.com/apache/camel/blob/master/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbProducer.java#L437

I'll have to try and dig into it a bit more.

[jamesnetherton]

It works when the camel component is compiled against the 4.x libs, otherwise we hit the NoSuchMethodError.

So I think we need a 4.x compatible camel component for this to work.

[loicmathieu]

@jamesnetherton we will need a coordinated release for Quarkus and Camel-Quarkus, but it's always the case right ?
So this will works if you do the same on your side and we both target the same release.

[jamesnetherton]

@loicmathieu yes, I created a Camel issue to track the upgrade work.

[gastaldi]

Is this PR OK to be merged?

[Postremus]

@jamesnetherton
I see that your camel pr is merged.
Can we upgrade to the 4.0 of the java driver in quarkus 1.4.0, or should we wait?

[jamesnetherton]

@ppalaga @lburgazzoli do you think this is ok to be merged? The Camel changes wont be available until 3.2.0 is released, so we'd be blocked upgrading Quarkus to 1.4.0 until it's available.

[jamesnetherton]

Let's not hold this back any longer. I'm ok for it to be merged 👍