Disable the Kafka Connector graceful shutdown in dev mode #18945

cescoffier · 2021-07-22T16:56:51Z

Also, propose an option to re-enable it.
The graceful shutdown waits until the inflight records have been processed and the offset committed to Kafka.
While this setting is highly recommended in production, in dev mode, it introduces slowness on shutdown.

quarkus-bot · 2021-07-22T17:12:40Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `40fc224`

Status	Name	Step	Test failures	Logs	Raw logs
✖	Initial JDK 11 Build	`Build`	⚠️ Check →	Logs	Raw logs

stuartwdouglas · 2021-07-23T05:38:26Z

...kus/smallrye/reactivemessaging/kafka/deployment/SmallRyeReactiveMessagingKafkaProcessor.java


-        discoverDefaultSerdeConfig(discoveryState, defaultConfigProducer);
+        if (launchMode.getLaunchMode() == LaunchMode.DEVELOPMENT) {


What about test mode as well? Will this potentially add extra time at the end of test runs?

Good idea. Should it be controlled by the same configuration property?

Right now it uses enableGracefulShutdownInDevMode, enableGracefulShutdownInDevAndTestMode is a bit long, but with autocompletion, it should not be a problem.

quarkus-bot · 2021-07-23T06:37:02Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `37804a3`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 16	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

✖ io.quarkus.it.resteasy.reactive.kotlin.ReactiveClientTest.testGetCountryByName - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.IllegalStateException: More than one default value for mp.messaging.incoming.countries-t1-in.graceful-shutdown was produced
	at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:820)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at...

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

✖ io.quarkus.it.resteasy.reactive.kotlin.ReactiveClientTest.testGetCountryByName - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.IllegalStateException: More than one default value for mp.messaging.incoming.countries-t1-in.graceful-shutdown was produced
	at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:820)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at...

...kus/smallrye/reactivemessaging/kafka/deployment/SmallRyeReactiveMessagingKafkaProcessor.java

Ladicek · 2021-07-23T07:44:01Z

Looks fine, barring the missing handling of @Channel injection points.

quarkus-bot · 2021-07-23T09:12:08Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `ae3a43f`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 16	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

✖ io.quarkus.it.resteasy.reactive.kotlin.ReactiveClientTest.testGetCountryByName - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.IllegalStateException: More than one default value for mp.messaging.incoming.countries-t1-in.graceful-shutdown was produced
	at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:820)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at...

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

✖ io.quarkus.it.resteasy.reactive.kotlin.ReactiveClientTest.testGetCountryByName - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.IllegalStateException: More than one default value for mp.messaging.incoming.countries-t1-in.graceful-shutdown was produced
	at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:820)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at...

quarkus-bot · 2021-07-23T14:22:51Z

This workflow status is outdated as a new workflow run has been triggered.

Failing Jobs - Building `d070c46`

Status	Name	Step	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 11	`Build`	Test failures	Logs	Raw logs
✖	JVM Tests - JDK 16	`Build`	Test failures	Logs	Raw logs

Full information is available in the Build summary check run.

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

✖ io.quarkus.it.resteasy.reactive.kotlin.ReactiveClientTest.testGetCountryByName - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.IllegalStateException: More than one default value for mp.messaging.incoming.countries-t1-in.graceful-shutdown was produced
	at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:820)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at...

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

✖ io.quarkus.it.resteasy.reactive.kotlin.ReactiveClientTest.testGetCountryByName - More details - Source on GitHub

java.lang.RuntimeException: 
java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors
	[error]: Build step io.quarkus.deployment.steps.ConfigGenerationBuildStep#generateConfigClass threw an exception: java.lang.IllegalStateException: More than one default value for mp.messaging.incoming.countries-t1-in.graceful-shutdown was produced
	at io.quarkus.deployment.steps.ConfigGenerationBuildStep.generateConfigClass(ConfigGenerationBuildStep.java:101)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at io.quarkus.deployment.ExtensionLoader$2.execute(ExtensionLoader.java:820)
	at io.quarkus.builder.BuildContext.run(BuildContext.java:277)
	at...

Also, propose an option to re-enable it. The graceful shutdown waits until the inflight records have been processed and the offset committed to Kafka. While this setting is highly recommended in production, in dev and test modes, it introduces slowness on shutdown.

cescoffier requested review from ozangunalp and Ladicek July 22, 2021 16:56

quarkus-bot bot added area/reactive-messaging area/smallrye labels Jul 22, 2021

cescoffier force-pushed the disable-kafka-graceful-shutdown-in-dev-mode branch from 40fc224 to e3880b0 Compare July 22, 2021 17:44

stuartwdouglas reviewed Jul 23, 2021

View reviewed changes

cescoffier force-pushed the disable-kafka-graceful-shutdown-in-dev-mode branch from e3880b0 to 37804a3 Compare July 23, 2021 06:05

stuartwdouglas approved these changes Jul 23, 2021

View reviewed changes

ozangunalp approved these changes Jul 23, 2021

View reviewed changes

Ladicek reviewed Jul 23, 2021

View reviewed changes

...kus/smallrye/reactivemessaging/kafka/deployment/SmallRyeReactiveMessagingKafkaProcessor.java Outdated Show resolved Hide resolved

cescoffier force-pushed the disable-kafka-graceful-shutdown-in-dev-mode branch from 37804a3 to ae3a43f Compare July 23, 2021 08:45

Ladicek approved these changes Jul 23, 2021

View reviewed changes

cescoffier force-pushed the disable-kafka-graceful-shutdown-in-dev-mode branch from ae3a43f to d070c46 Compare July 23, 2021 13:49

cescoffier force-pushed the disable-kafka-graceful-shutdown-in-dev-mode branch from d070c46 to 22bba29 Compare July 23, 2021 14:54

stuartwdouglas merged commit 0d70cba into quarkusio:main Jul 24, 2021

quarkus-bot bot added this to the 2.2 - main milestone Jul 24, 2021

stuartwdouglas added the triage/backport? label Jul 24, 2021

cescoffier deleted the disable-kafka-graceful-shutdown-in-dev-mode branch August 1, 2021 07:31

gsmet modified the milestones: 2.2 - main, 2.1.1.Final Aug 3, 2021

gsmet added triage/backport? and removed triage/backport? labels Aug 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Disable the Kafka Connector graceful shutdown in dev mode #18945

Disable the Kafka Connector graceful shutdown in dev mode #18945

cescoffier commented Jul 22, 2021

quarkus-bot bot commented Jul 22, 2021 •

edited

Loading

Failing Jobs - Building `40fc224`

stuartwdouglas Jul 23, 2021

cescoffier Jul 23, 2021

cescoffier Jul 23, 2021

quarkus-bot bot commented Jul 23, 2021 •

edited

Loading

Failing Jobs - Building `37804a3`

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

Ladicek commented Jul 23, 2021

quarkus-bot bot commented Jul 23, 2021 •

edited

Loading

Failing Jobs - Building `ae3a43f`

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

quarkus-bot bot commented Jul 23, 2021 •

edited

Loading

Failing Jobs - Building `d070c46`

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard


		discoverDefaultSerdeConfig(discoveryState, defaultConfigProducer);
		if (launchMode.getLaunchMode() == LaunchMode.DEVELOPMENT) {

Disable the Kafka Connector graceful shutdown in dev mode #18945

Disable the Kafka Connector graceful shutdown in dev mode #18945

Conversation

cescoffier commented Jul 22, 2021

quarkus-bot bot commented Jul 22, 2021 • edited Loading

Failing Jobs - Building 40fc224

stuartwdouglas Jul 23, 2021

Choose a reason for hiding this comment

cescoffier Jul 23, 2021

Choose a reason for hiding this comment

cescoffier Jul 23, 2021

Choose a reason for hiding this comment

quarkus-bot bot commented Jul 23, 2021 • edited Loading

Failing Jobs - Building 37804a3

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

Ladicek commented Jul 23, 2021

quarkus-bot bot commented Jul 23, 2021 • edited Loading

Failing Jobs - Building ae3a43f

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

quarkus-bot bot commented Jul 23, 2021 • edited Loading

Failing Jobs - Building d070c46

Test Failures

⚙️ JVM Tests - JDK 11 #

📦 integration-tests/resteasy-reactive-kotlin/standard

⚙️ JVM Tests - JDK 16 #

📦 integration-tests/resteasy-reactive-kotlin/standard

quarkus-bot bot commented Jul 22, 2021 •

edited

Loading

Failing Jobs - Building `40fc224`

quarkus-bot bot commented Jul 23, 2021 •

edited

Loading

Failing Jobs - Building `37804a3`

quarkus-bot bot commented Jul 23, 2021 •

edited

Loading

Failing Jobs - Building `ae3a43f`

quarkus-bot bot commented Jul 23, 2021 •

edited

Loading

Failing Jobs - Building `d070c46`