quarkus:dev with quarkus-jdbc-d2 ignores db2jcc_license_cisuz dependency

[kraeftbraeu]

Describe the bug

Trying out quarkus-jdbc-db2 to connect to our mainframe db2 I found a weird behaviour:

After adding the dependencies quarkus-jdbc-db2 and my db2jcc_license_cisuz and running mvn package I can run the quarkus-run.jar and it connects to the db.

But when I start quarkus:dev and query the db I get a "missing license" error (see below).

Expected behavior

I expect quarkus:dev to use my db2jcc_license_cisuz dependency as it works with mvn install + run jar

Actual behavior

When I start quarkus:dev and trigger a db2 call I receive the log

Verbindung zum Datenserver ist fehlgeschlagen. Die Lizenz für IBM Data Server for JDBC and SQLJ war ungültig 
oder nicht für das DB2 für z/OS-Subsystem aktiviert. Wenn Sie eine direkte Verbindung zum 
Datenserver herstellen und DB2 Connect Unlimited Edition für System z verwenden, führen Sie 
den Aktivierungsschritt aus, indem Sie das Aktivierungsprogramm im Lizenzaktivierungskit ausführen.  
Wenn Sie eine andere Edition von DB2 Connect verwenden, suchen Sie die Lizenzdatei 
db2jcc_license_cisuz.jar im Lizenzaktivierungskit und befolgen Sie die Installationsanweisungen, 
um die Lizenzdatei in den Klassenpfad aufzunehmen. ERRORCODE=-4230, SQLSTATE=42968

To Reproduce

• mvn 3.6.3
• Java version: 11.0.10, vendor: Azul Systems, Inc.
• quarkus 1.13.2.Final (not native)
• use dependency quarkus-jdbc-db2
• add container-license-acceptance.txt to accept license
• add a valid db2jcc_license_cisuz dependency
• run mvn clean quarkus:dev

Configuration

%prod.quarkus.datasource."db2".jdbc.url=${DB2_URL}
%prod.quarkus.datasource."db2".username=${DB2_HANDOVER_USER}
%prod.quarkus.datasource."db2".password=${DB2_HANDOVER_PWD}
%dev.quarkus.datasource."db2".jdbc.url=${DB2_URL}
%dev.quarkus.datasource."db2".username=${DB2_HANDOVER_USER}
%dev.quarkus.datasource."db2".password=${DB2_HANDOVER_PWD}
%test.quarkus.datasource."db2".devservices=true

Screenshots

(If applicable, add screenshots to help explain your problem.)

Environment (please complete the following information):

Output of uname -a or ver

Linux 5.8.0-48-generic #54~20.04.1-Ubuntu SMP Sat Mar 20 13:40:25 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

openjdk version "11.0.10" 2021-01-19 LTS
OpenJDK Runtime Environment Zulu11.45+28-SA (build 11.0.10+9-LTS)
OpenJDK 64-Bit Server VM Zulu11.45+28-SA (build 11.0.10+9-LTS, mixed mode)

GraalVM version (if different from Java)

I don't use native

Quarkus version or git rev

quarkus 1.13.2.Final

Build tool (ie. output of mvnw --version or gradlew --version)

Apache Maven 3.6.3
Maven home: /usr/share/maven
Java version: 11.0.10, vendor: Azul Systems, Inc., runtime: ~/apps/zulu11.45.28-sa-jdk11.0.10-linux_x64
Default locale: de_DE, platform encoding: UTF-8
OS name: "linux", version: "5.8.0-48-generic", arch: "amd64", family: "unix"

Additional context

https://www.ibm.com/support/pages/apar/IC82094 (Local fix 1) says I have to add a db2jcc_license_cisuz.jar to the classpath. This works when I build the app with mvn clean install and run the quarkus-run.jar. With mvn quarkus:dev that doesn't work.

[quarkus-bot]

/cc @mswatosh

[teksupp12]

I am also facing the same issue. Could someone please help!

I expect quarkus:dev to use my db2jcc_license_cisuz dependency as it works with mvn install + run jar

It worked fine till 1.13.* version. However after upgrading quarkus version to 2.0.* I am seeing this issue :(

[Sanne]

hi @mswatosh - could you have a look at this please?

[mswatosh]

@Sanne Let me know if I'm not reading this right, but it sounds like this is a classloading issue with Dev mode? Since it's working with mvn package

[Sanne]

hi Mark! Yes I agree, it's most likely related to classloaders. The puzzling aspect is that such things generally work fine, so it would be useful to look at what the driver code is doing which is particular in this case.

…

On Wed, 17 Nov 2021, 15:04 Mark Swatosh, ***@***.***> wrote: @Sanne <https://github.com/Sanne> Let me know if I'm not reading this right, but it sounds like this is a classloading issue with Dev mode? Since it's working with mvn package — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#16613 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAKMTMTCQHJNN2G5VW2MYTUMO765ANCNFSM43FDKGUQ> .

[mswatosh]

I just realized this is the about JDBC driver (the JCC driver) not the reactive driver. Unfortunately I don't have access to the JCC driver, it's maintained by a third party company now. Based on the license file however, I can guess that all the JCC driver is doing is checking if the classes provided by the license jar are on the classpath, and if so then it allows the connection. So I believe this error is just the result of the JCC driver not being able to find the classes in the license jar.

[teksupp12]

Thank you for the responses. Could we expect any fixes for this in upcoming releases?
@Sanne @mswatosh

[srigano]

Hi,
"o I believe this error is just the result of the JCC driver not being able to find the classes in the license jar." -> yes
I opened another issue : #21993.

If we have a valid db2jcc_license_cisuz dependency, what should we add in the container-license-acceptance.txt file ??
Thanks.

[srigano]

Nota: i'm using the latest quarkus platform : 2.5.1.Final.

[teksupp12]

@mswatosh @Sanne. Could you please help us here?

[Sanne]

hi, yes sorry about this. I've emailed @mswatosh directly as I suspect he didn't see the notifications here. He's in touch with Rocket Soft, the DB2 driver maintainers - I hope he'll be able to update us soon.

[mswatosh]

I reached out to the team that owns the JCC driver and they're looking into addressing this issue on their end. Unfortunately, their next fixpack release won't be until June 2022. They said they can send me a driver to test once the fix is ready, but I'm not sure if that would be production ready/how we would consume it in Quarkus, since I'm guessing it currently pulls an official driver from Maven Central.

[Sanne]

thanks for the update @mswatosh. Do you happen to know why it's not finding the license? Could Rocket Soft people perhaps suggest what we could do differently in Quarkus?

[mswatosh]

Yes, it's due to this issue: #2809
Honestly, fixing this in the JCC driver is really addressing a symptom, not the cause of the issue. The reason for not fixing in that issue was:

I am closing this, as there are no plans to enable Class.forName() to work for every situation (this can only be done with a flat classpath, which means no hot reload)

Which seems to be accepting that some libraries won't work the same in dev mode as production. If hot reload is the issue, then it would be nice to have an option to specify a library as not reloadable, and allow Class.forName() to work. Any library that needs to be hot reloaded, should be easy fix, and any library that can't be fixed, shouldn't need to be hot reloaded.

[Sanne]

Which seems to be accepting that some libraries won't work the same in dev mode as production.

To be fair, most libraries need to be able to work within various classloading models. I had never found in practice a library that would require a flat model, except occasional bugs of course, but never an intentional requirement for this.

@stuartwdouglas I vaguely remember you had to do some classloader trick specifically to workaround JDBC driver needs. Could you point us to it? Could be related, or perhaps a workaround could be driven by that ad-hoc code?

[stuartwdouglas]

Can you try:
quarkus.class-loading.parent-first-artifacts={groupId}:db2jcc_license_cisuz

Where {groupId} is whatever the group id of the jar is.

[srigano]

it does not works. i tried:

quarkus.class-loading.parent-first-artifacts=db2jccdriver:db2jcc_license_cisuz
and in my pom.xml file:

<dependency>
  <groupId>db2jccdriver</groupId>
  <artifactId>db2jcc_license_cisuz</artifactId>
  <version>10.5.0.8</version>
</dependency>

i also tried to add the .jar in the src/main/resources folder
And as the jar file name is db2jcc_license_cisuz-10.5.0.8.jar i also tried this property value:

quarkus.class-loading.parent-first-artifacts=db2jccdriver:db2jcc_license_cisuz-10.5.0.8

Still the same issue

but if i package my application and deploy it in a kubernetes its works well. It really appears to be an issue with the quarkus dev mode.

[stuartwdouglas]

What is actually in this jar? Is it just resource files? And do we know how it is loaded by the driver?

There is probably an existing solution as the class loading model is quite flexible, its just that without knowing exactly what it is trying to do and with no reproducer I am not sure what it would be.

[srigano]

3 class files and a manifest:

[srigano]

when my application is mvn packaged, it works well.
So it really seems to be a class-loading problem

[srigano]

any idea ?

[srigano]

Hi,

@quarkus Team: could you please look at this bug ?
Is there any way to fix it ? Thanks.

[stuartwdouglas]

@srigano I just noticed in the description you are using Quarkus 1.x which is not going to get updates anyway, can you test with 2.7.1 when it is released (tomorrow I think) and the parent first config that I mentioned above? 2.7.1 contains some parent first fixes which might make a difference.

[srigano]

It's the same with
Quarkus 2.6.x
Quarkus 2.7.0
Tested today.
Sure I ll test as soon as this version is available.

[stuartwdouglas]

I think this PR might help, which is going into 2.7.1: #23357

[gsmet]

2.7.1.Final will be released tomorrow.

…

On Mon, Feb 7, 2022 at 11:03 PM Stuart Douglas ***@***.***> wrote: I think this PR might help, which is going into 2.7.1: #23357 <#23357> — Reply to this email directly, view it on GitHub <#16613 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJYOBIPLG3V52DCOLCZ6KDU2A6SXANCNFSM43FDKGUQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[srigano]

Thank you very much
I will test this issue as soon as possible tomorrow or the day after and let you know.

[kraeftbraeu]

Sorry for not participating the communication during implementation, I had to much other stuff to do...

But now I managed to update to quarkus 2.7.1 and check the issue with additional prop

%dev.quarkus.class-loading.parent-first-artifacts={groupId}:db2jcc_license_cisuz

It works like a charm now, thank you @stuartwdouglas for the fix and to all the others for pushing that topic!

It is up to me to close the issue now, right?

[bvahdat]

@Sanne @stuartwdouglas @gsmet

I am running into the exact same problem using 2.7.1.Final because my config source is not Properties file but YML under the config folder.

If I switch to use Properties under src/main/resources/application.properties then it works but providing the following in config/application-dev.yml doesn´t work:

quarkus:
  class-loading:
    parent-first-artifacts: com.ibm.db2.jcc:db2jcc_license_cisuz

Do you want me to create another issue or should we reopen this one?

[srigano]

There is another issue : 23357. Le ven. 18 févr. 2022 à 17:32, Babak Vahdat ***@***.***> a écrit :

…

@Sanne <https://github.com/Sanne> @stuartwdouglas <https://github.com/stuartwdouglas> @gsmet <https://github.com/gsmet> I am running into the exact same problem using 2.7.1.Final because my config source is not Properties file but YML under the config folder. If I switch to use Properties under src/main/resources/application.properties then it works but providing the following in config/application-dev.yml doesn´t work: quarkus: class-loading: parent-first-artifacts: com.ibm.db2.jcc:db2jcc_license_cisuz Do you want me to create another issue or should we reopen this one? — Reply to this email directly, view it on GitHub <#16613 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKNELQTSMTYBYLEUFVHMLN3U3ZYA3ANCNFSM43FDKGUQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[bvahdat]

@srigano
#23357 fixed your issue but not mine, so why I am asking here. IMHO this reported issue here needs to be reopened as the provided fix by #23357 doesn´t work in general, e.g. when using config/application-dev.yml.

[Sanne]

@bvahdat interesting - could you open a different issue please?

If it works with application.properties then it suggests the specific issue with classloading of the license is resolved, there most likely is some issue in how the .yml file is interpreted - which would be a bug in configuration parsing, not specific to DB2.

[Sanne]

ah you did it already - perfect thanks!

[stuartwdouglas]

It's because class loading config is special, its not the same as regular config and there is some hacky code that reads it from applications.properties in the early stages of bootstrap / inside the maven plugin itself.

[srigano]

hi,
it works still 2.7.1.Final with :
quarkus.class-loading.parent-first-artifacts=db2jccdriver:db2jcc_license_cisuz

Thank you very much