Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Quarkus Mongodb change stream resume token always null in native #17878

Closed
acuiuli opened this issue Jun 13, 2021 · 9 comments · Fixed by #18061
Closed

Quarkus Mongodb change stream resume token always null in native #17878

acuiuli opened this issue Jun 13, 2021 · 9 comments · Fixed by #18061
Labels
area/mongodb env/windows Impacts Windows machines kind/bug Something isn't working
Milestone

Comments

@acuiuli
Copy link

acuiuli commented Jun 13, 2021

Describe the bug

I'm building a quarkus app that uses the mongodb change streams functionality with the reactive client.

If I start the application locally from Intellij everything works fine but when I build the native app and try to save the resume token it's always null.

I use this command to build the native app since I need to build it locally on windows and delpoy it on heroku:

mvn package -Pnative -Dquarkus.native.container-build=true -Dquarkus.native.builder-image=quay.io/quarkus/ubi-quarkus-native-image:21.0.0-java11

This is the class and method where I initialize the change stream and start the watch on the collection

@Inject
ReactiveMongoClient mongoClient;

private void initOrderStream() {
    ReactiveMongoDatabase database = mongoClient.getDatabase("database");
    ReactiveMongoCollection<Order> dataCollection = database.getCollection("order", Order.class);
    ChangeStreamOptions options = new ChangeStreamOptions().fullDocument(FullDocument.UPDATE_LOOKUP);


    List<Bson> pipeline = Collections.singletonList(
            Aggregates.match(
                    Filters.and(
                            Filters.eq("operationType", "insert")
                    )
            )
    );

    Multi<ChangeStreamDocument<Order>> publisher = dataCollection.watch(pipeline, Order.class, options);
    publisher.subscribe().with(eventListener.getOrderListener());
}

And this is the listener method

public Consumer<ChangeStreamDocument<Order>> getOrderListener() {
    return message -> {
        logger.info("Saving resume token for message " + message);
        EventConfig eventConfig = new EventConfig();
        eventConfig.setType(eventType);
        eventConfig.setResumeToken(message.getResumeToken().getString("_data"));
        EventConfig.persistOrUpdate(eventConfig);

	    Order order = message.getFullDocument();
	    saveEvent(order);
    };
}

The problem is in the message.getResumeToken() method, if I start the application non native than the resume token is present and I can save it without problems, but when I build it native and put it in a container the resume token is always null.

Without resume token the change streams are almost useless since Iit's not possible to recover missing events if a problem occurs or appplication needs to be restarted.

Expected behavior

The resume token using the getResumeToken method of the ChangeStreamDocument class should not return null in native mode.

Actual behavior

The resume token using the getResumeToken method of the ChangeStreamDocument class is always null in native mode.

To Reproduce

Steps to reproduce the behavior:

  1. Create a quarkus app
  2. Connect it to mongodb
  3. Initialize change stream on a collection using the code mentioned above
  4. Package the application in native mode using the maven command above
  5. Start the application
  6. Insert a document in the collection attached to the watcher and receive a null pointer exception because the resume token is null

Environment (please complete the following information):

Output of java -version

openjdk version "11.0.10" 2021-01-19
OpenJDK Runtime Environment GraalVM CE 21.0.0 (build 11.0.10+8-jvmci-21.0-b06)
OpenJDK 64-Bit Server VM GraalVM CE 21.0.0 (build 11.0.10+8-jvmci-21.0-b06, mixed mode, sharing)

Quarkus version or git rev

1.13.6.Final

@acuiuli acuiuli added the kind/bug Something isn't working label Jun 13, 2021
@quarkus-bot
Copy link

quarkus-bot bot commented Jun 13, 2021

/cc @evanchooly, @loicmathieu

@quarkus-bot quarkus-bot bot added area/mongodb env/windows Impacts Windows machines labels Jun 13, 2021
@loicmathieu
Copy link
Contributor

@loicmathieu
Copy link
Contributor

@AndreaCuiuli can you provide a JUNit test case inside your reproducer project or at least some step to reproduce the issue ?

@loicmathieu
Copy link
Contributor

@AndreaCuiuli I successfully reproduce the error, the com.mongodb.client.model.changestream.ChangeStreamDocument needs to be added for reflection for GraalVM native images.
I'll provide a fix for it.

Meanwhile, you can workaround the issue by creating a class that will register the class, for eg:

import com.mongodb.client.model.changestream.ChangeStreamDocument;
import io.quarkus.runtime.annotations.RegisterForReflection;

@RegisterForReflection(targets = ChangeStreamDocument.class)
public class ReflectionConfig {
}

@loicmathieu
Copy link
Contributor

@AndreaCuiuli in fact this has already been fixed in 1.13.5, can you try with the latest 1.13 version, should be 1.13.7.Final, and confirmed it works?

@acuiuli
Copy link
Author

acuiuli commented Jun 21, 2021

@loicmathieu Thanks for the update, I will try and let you know!

@acuiuli
Copy link
Author

acuiuli commented Jun 22, 2021

Hi @loicmathieu, I've made a test with the version 1.13.7.Final but the resumeToken is still null.
This is the log of the ChangeStreamDocument that I receive:

[com.tea.eve.eve.EventListener] (async-channel-group-0-handler-executor) Saving resume token for message ChangeStreamDocument{ operationType=OperationType{value='delete'}, resumeToken=null, namespace=teascannerDb.mongoEvent, destinationNamespace=null, fullDocument=null, documentKey={"_id": {"orderId": "60be8d8ba1d87b4bf012445c", "type": "WAITING_ORDER_REMINDER"}}, clusterTime=Timestamp{value=6976499491255877633, seconds=1624342867, inc=1}, updateDescription=null, txnNumber=null, lsid=null}

As you can see even with this version it's null.

@acuiuli
Copy link
Author

acuiuli commented Jun 22, 2021

The workaround that you suggested worked fine though.

I created this class

import com.mongodb.client.model.changestream.ChangeStreamDocument;
import io.quarkus.runtime.annotations.RegisterForReflection;

@RegisterForReflection(targets = ChangeStreamDocument.class)
    public class ReflectionConfig {
}

And the resume token is not null anymore

@loicmathieu
Copy link
Contributor

@AndreaCuiuli I may have a any why it's happening, meanwhile you can still use the workaround I suggested.

loicmathieu added a commit to loicmathieu/quarkus that referenced this issue Jun 22, 2021
loicmathieu added a commit to loicmathieu/quarkus that referenced this issue Jun 22, 2021
@quarkus-bot quarkus-bot bot added this to the 2.1 - main milestone Jun 22, 2021
@gsmet gsmet modified the milestones: 2.1 - main, 2.0.0.Final Jun 22, 2021
gsmet pushed a commit to gsmet/quarkus that referenced this issue Jun 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/mongodb env/windows Impacts Windows machines kind/bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants