PanacheMongoRepository should always support @Transactional regardless if Mongo is clustered

[dhoffer]

Description

We would like all of our JAX-RS services that use a PanacheMongoRepository<> datasource to use @transactional(Transactional.TxType.REQUIRED).

This works fine if the app is deployed in a clustered Mongo DB environment, however when deployed in a single Mongo instance environment it causes runtime errors saying it can't handle/create transactions.

So our only workaround was to remove all usages of @transactional(Transactional.TxType.REQUIRED) which then does work in both environments.

We would like to see Quarkus improved so that we can use @transactional where it can be supported and just warn in single Mongo instance environments where it can't be supported. Even better would be to support @transactional in single Mongo instances as well.

Implementation ideas

No response

[quarkus-bot]

/cc @FroMage, @evanchooly, @loicmathieu

[loicmathieu]

Even better would be to support @transactional in single Mongo instances as well.

This can only be done at MongoDB server side, this is a limitation of MongoDB that mandates a replica set for transaction to work.

warn in single Mongo instance environments where it can't be supported

What we can do is to provide a configuration option to disable transaction support globaly, so when running in a single instance server you can disable it easily.

[dhoffer]

@loicmathieu Yes the config option of being able to disable transaction support would be great. Then we can use Transactions in the code base but disable via config when installed in non clustered Mongo environments.

[evanchooly]

Or just run single node replica sets. You need that for backups anyway unless you just have mongodump cron'd to dump the entire database at regular intervals. Adding this property seems counterproductive to me.

[loicmathieu]

@evanchooly if a single node replicaset allows to use transactions, so we may ned to support it on devservices, so transactions can be used on tests and dev more.

@dhoffer is your use case for local developement and tests ?

[evanchooly]

It does. It's how I test morphia.

[dhoffer]

@loicmathieu @evanchooly Our MongoDB use case is in Production on a AWS Snowball. It has limited hardware so we can't run a cluster but I can ask if we can run as a single node replica sets, I don't know the answer to that question. We normally do run a cluster of 3-9? MongoDBs but can't do that on the Snowball (and the data load will be much lighter).

[evanchooly]

@evanchooly if a single node replicaset allows to use transactions, so we may ned to support it on devservices, so transactions can be used on tests and dev more.

If flapdoodle doesn't support replSets, bottlerocket does and is pretty trivial to use. Supports sharded clusters, too, for testing certain scenarios.

[csmuller]

if a single node replicaset allows to use transactions, so we may ned to support it on devservices, so transactions can be used on tests and dev more.

@loicmathieu I'm having the same problem as OP when using the devservices. I guess the devservices spins up a standalone instance. So, having it spin up a single-node replica set would be much appreciated.

[loicmathieu]

@csmuller I checked and testcontainers MongoDBContainer which is used by our devservices do start a single instance replicaset and do support transactions (at least with version 4.4).

So using devservices, transactions will be possible. I'll review our documentation but as devservices is the prefered way to run MongoDB on dev mode and test and it setup a one instance replicaset I consider this issue as already implemented.

As @evanchooly pointed out, one-node replicaset is possible so it's the way to have single-node transaction support.

[csmuller]

@loicmathieu Thanks for the speedy answer. Indeed, it worked with the mongo:4.4 docker image.
Using a mongo version starting at 6 fails, because the MongoDBContainer expects the "old" mongo cmd line tool but should use the new mongosh.
Just mentioning that here for completeness. Not something that Quarkus has control over.

[loicmathieu]

@csmuller this should be discussed with the testcontainers team, maybe open an issue in their repository.

[loicmathieu]

@dhoffer as a single MongoDB instance can be setup as a replicaset and this is what our devserices provides can we close this issue ? I updated the documentation to better describe this, see #27781

[dhoffer]

@loicmathieu @csmuller @evanchooly I don't have a great/perfect answer for you on this. Your solution seems reasonable/logical to me but the person that does our MongoDB installs feels that a one replicaset install is a waste of resources (and its on a resource constrained system) so doesn't like the idea of installing that way. He did say its technically possible he just doesn't want the resource overhead I guess. Our timeline is short so we probably can't switch to one replicaset install at this time but perhaps at a later date we can, not sure.

[evanchooly]

I don't know of any resource which suggests that a single node replica set consumes significantly more resources. it does have an oplog which will, of course, consume some resources but will it be noticeable? I haven't found that to be the case, personally. The oplog can be sized down to whatever you want it to be, though, and offset some of that.

[loicmathieu]

As there is a solution to use transactions with a one node replicaset cluster I considere this issue as resolved. Using transaction always comes with a cost so it's natural that using a one node replicaset cluster has an overhead compared to a non-replicaset custer but an oplog that will not be replicated should have very low overhead.