DB2 - sending back 500: java.lang.IllegalStateException: Found unknown codepoint: 0x1153 / 4435

[cescoffier]

Context

This is a copy of quarkusio/quarkus#14268.

Describe the bug

While I am trying to fetch a record from a DB2 Table using a reactive datasource, I am getting the following exception. The exception doesn't tell me much what is causing the issue. However a similar code on the same codebase when used to query on a PostgreSQL Table using a Postgres reactive datasource, it is working fine.

2021-01-12 17:50:52,656 ERROR [org.jbo.res.res.i18n] (vert.x-eventloop-thread-14) RESTEASY002020: Unhandled asynchronous exception, sending back 500: java.lang.IllegalStateException: Found unknown codepoint: 0x1153 / 4435
at io.vertx.db2client.impl.drda.DRDAResponse.throwUnknownCodepoint(DRDAResponse.java:847)
at io.vertx.db2client.impl.drda.DRDAConnectResponse.parseVALNSPRM(DRDAConnectResponse.java:220)
at io.vertx.db2client.impl.drda.DRDAConnectResponse.parseCommonError(DRDAConnectResponse.java:118)
at io.vertx.db2client.impl.drda.DRDAConnectResponse.parseSECCHKreply(DRDAConnectResponse.java:1047)
at io.vertx.db2client.impl.drda.DRDAConnectResponse.readSecurityCheck(DRDAConnectResponse.java:50)
at io.vertx.db2client.impl.codec.InitialHandshakeCommandCodec.decodePayload(InitialHandshakeCommandCodec.java:102)
at io.vertx.db2client.impl.codec.DB2Decoder.decodePayload(DB2Decoder.java:80)
at io.vertx.db2client.impl.codec.DB2Decoder.decode(DB2Decoder.java:53)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)

Expected behavior

The exception doesn't tell me much about the issue that is causing it to fail. A similar code when run on the PostgreSQL Table there is no issue. Ideally the behavior for both PostgreSQL and DB2 driver should be consistent.

Actual behavior

A similar code when I am running using a PostgreSQL reactive datasource I m not facing any issue. So the expectation is the behavior should be same for DB2 Datasource as well.

To Reproduce

I cannot share the actual codebase as its against my employer's policy, so I m providing a sample code in Kotlin that is using the DB2Pool. A similar code is working with PgPool.

import io.vertx.mutiny.db2client.DB2Pool
...
class SampleRepo {
    companion object {
        private val table = "table_name"
        private val schema = "schema_name"

        fun find(db2Pool: DB2Pool, str: String): Uni<SampleDO?> {
            val SELECT_ONE = "SELECT * FROM ${schema}.${table} WHERE str=$str"
            return db2Pool.preparedQuery(SELECT_ONE)
                .execute()
                .onItem().transform { it.iterator() }
                .onItem().transform { rowIterator ->
                    if (rowIterator.hasNext())
                        return@transform rowMap2SampleDO(rowIterator.next())
                    else null
                }
        }

        private fun rowMap2SampleDO(row: Row): SampleDO {
            return SampleDO(
                row.getString("xxx1"),
                row.getInteger("xxx2"),
                row.getInteger("xxx3"),
                row.getString("xxx4"))
       }
   }
}

Configuration

quarkus.datasource.reactive=true

quarkus.datasource.pg.db-kind=postgresql
quarkus.datasource.pg.username=xxx
quarkus.datasource.pg.password=xxx
quarkus.datasource.pg.reactive.url=postgresql://xxx

quarkus.datasource.db2.db-kind=db2
quarkus.datasource.db2.username=xxx
quarkus.datasource.db2.password=xxx
quarkus.datasource.db2.reactive.url=vertx-reactive:db2://xxx

Environment:

Output of uname -a or ver: Darwin xxx 19.6.0 Darwin Kernel Version 19.6.0: Tue Nov 10 00:10:30 PST 2020; root:xnu-6153.141.10~1/RELEASE_X86_64 x86_64
Output of java -version: I m using Kotlin 1.3.72

 ~$ java -version
openjdk version "11.0.2" 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
GraalVM version (if different from Java): NA
Quarkus version or git rev: 1.10.2.Final

Build tool (ie. output of mvnw --version or gradlew --version):

~$ mvn --version
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /xxx/xxx/xxx/maven/3.6.3_1/libexec
Java version: 11.0.2, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.15.7", arch: "x86_64", family: "mac"

Let me know if you need any additional details. I can provide the same here.

Note: Some of the information that cannot be shared here, has been masked as xxx.

[pmlopes]

@cescoffier is the data being fetched binary? If that is the case, probably the use of String which implies parsing the data as UTF-8 is the cause? Or maybe the data is in a different character encoding?

[cescoffier]

@arunav-b can you reply to @pmlopes ? ^

[mswatosh]

That error path is particularly bad. Based on where we are in the code and the codepoint that's returned (0x1153, which is Server diagnostic info) there is likely something that needs to be fixed in the statement being executed. The DB2 driver should be returning a sqlstate of 58016, which is "Parameter Not Supported." I believe the server diagnostic info may also be providing more information, which should be returned to the user.

I'll look into making these improvements, but based on where this is in the code, I think the DB2 server has an issue with the statement.

[mswatosh]

Actually it looks like I was wrong, this isn't an issue with the statement, but with authentication:
at io.vertx.db2client.impl.drda.DRDAConnectResponse.parseSECCHKreply(DRDAConnectResponse.java:1047)
at io.vertx.db2client.impl.drda.DRDAConnectResponse.readSecurityCheck(DRDAConnectResponse.java:50)

I'll do some poking around to see if I can recreate this, but it may not be easy to recreate.

[mswatosh]

I tried some basic security failure scenarios but wasn't able to recreate the error. @arunav-b can you tell us what you're setting for DB2ConnectOptions? I'm wondering if this is an SSL or TLS failure path

[peterl1084]

Having exactly the same issue. @mswatosh can I provide you with any details that'd help you. I'm seeing exactly the same parseSECCHKreply after which parseCommonError takes it to parseVALNSPRM. After first pass it fails during the second as peekCP is 4435. The first reply to parseSECCHKreply is 4690.

Using Quarkus 2.7.1 with vertx-db2-client-4.2.4.

All options are default and configuration is as follows:

datasource: db-kind: db2 username: xxx password: xxx reactive: url: db2://ip-addess/db-name

Only special case I think is that the url has database defined with IP

[mswatosh]

@peterl1084 I just tried to recreate this (using Quarkus this time instead of directly in Vert.x) but even using the IP instead of the hostname I'm still getting a successful connection. Two things:

1. Which version of DB2 are you using? (e.g. 10.5 on Linux)
2. Are you able to successfully connect using the JDBC driver instead of the reactive driver? If not, it might at least provide a better error message.

[peterl1084]

@mswatosh according to the DBA the particular DB2 instance is V12 function level 500 running in z/OS. I'll try with regular JDBC.

[mswatosh]

I was worried it might be Db2 z/OS, which explains why I can't recreate it. That's the one Db2 platform I don't have access to right now, but I've reached out to see if I can get a Db2 z/OS instance to attempt to recreate this with.

[peterl1084]

@mswatosh JDBC works.

According to the DBA the reactive driver version never reaches the DB, at least on the level that any query logging would get done, nothing appears in the trace logs for reactive connections.

[mswatosh]

Thanks for checking that, now we know this isn't just a bad error path it's an actual defect. Your DBA is right, we're not even getting past the initial security check here, so there won't be anything in the query logs. Hopefully I'll have a z/OS system to try to recreate this next week.

…

On Fri, Feb 18, 2022, 3:24 AM Peter Lehto ***@***.***> wrote: @mswatosh <https://github.com/mswatosh> JDBC works. According to the DBA the reactive driver version never reaches the DB, at least on the level that any query logging would get done, nothing appears in the trace logs for reactive connections. — Reply to this email directly, view it on GitHub <#1131 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFCMJM65AZGF6725CQIKPZ3U3YF43ANCNFSM5NRSQGLQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you were mentioned.Message ID: ***@***.***>

[swetapatra]

Hi Team,is there an update on this issue?

[mswatosh]

I got a contact in Db2/z but didn't get a system yet since the team was swamped at the time. I just sent a follow up so hopefully I can investigate this soon.

[swetapatra]

@mswatosh Thank you for the update.

[swetapatra]

@mswatosh are there any updates on this?As this is a blocker for one of our customers.Kindly let us know

[swetapatra]

@ALL can anyone else please help me with an update,as this is a blocker for one of our customers.

[vietj]

@mswatosh ping

[mswatosh]

@swetapatra I still haven't gotten access to a Db2/z system to recreate this on. I'll push a little more on the Db2/z team to get me access to a system.