vertx-sql-client icon indicating copy to clipboard operation
vertx-sql-client copied to clipboard
verified publisher icon eclipse-vertx

DB2 query throws IndexOutOfBoundsException

Open gbadner opened this issue 4 years ago • 14 comments

Version

4.1.3

Context

Using DB2, the following query:

select table_name as TABLE_NAME, column_name as COLUMN_NAME,  
             case when data_type = 'CHARACTER VARYING' then 'VARCHAR' else data_type end as TYPE_NAME, 
             null as COLUMN_SIZE, null as DECIMAL_DIGITS, is_nullable as IS_NULLABLE, null as DATA_TYPE 
from sysibm.columns 
where true  and table_catalog = ? 
order by table_catalog, table_schema, table_name, column_name, ordinal_position

throws:

 java.lang.IndexOutOfBoundsException: readerIndex(32741) + length(17) exceeds writerIndex(32757): ReadOnlyByteBuf(ridx: 32741, widx: 32757, cap: 32757/32757, unwrapped: PooledSlicedByteBuf(ridx: 0, widx: 32757, cap: 32757/32757, unwrapped: PooledUnsafeDirectByteBuf(ridx: 32860, widx: 32860, cap: 65536)))

See below for full stacktrace.

Do you have a reproducer?

No, but it should be easy to reproduce by adding the query to an existing test, since it only relies on sysibm.columns (DB2's version of information_schema).

We can provide a reproducer using Hibernate ORM and Reactive PRs. Please let me know if you'd like us to do that.

Steps to reproduce

  1. ...
  2. ...
  3. ...
  4. ...

Extra

  • Fedora
  • JDK 1.8

Full stacktrace:

java.util.concurrent.CompletionException: java.lang.IndexOutOfBoundsException: readerIndex(32741) + length(17) exceeds writerIndex(32757): ReadOnlyByteBuf(ridx: 32741, widx: 32757, cap: 32757/32757, unwrapped: PooledSlicedByteBuf(ridx: 0, widx: 32757, cap: 32757/32757, unwrapped: PooledUnsafeDirectByteBuf(ridx: 32860, widx: 32860, cap: 65536)))
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292) ~[?:1.8.0_302]
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308) ~[?:1.8.0_302]
        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607) ~[?:1.8.0_302]
        at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591) ~[?:1.8.0_302]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) ~[?:1.8.0_302]
        at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) ~[?:1.8.0_302]
        at io.vertx.core.Future.lambda$toCompletionStage$2(Future.java:362) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureImpl$3.onFailure(FutureImpl.java:153) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:75) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:230) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.tryFail(PromiseImpl.java:23) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.sqlclient.impl.QueryResultBuilder.tryFail(QueryResultBuilder.java:118) ~[vertx-sql-client-4.1.3.jar:4.1.3]
        at io.vertx.core.Promise.fail(Promise.java:89) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.Promise.handle(Promise.java:53) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.Promise.handle(Promise.java:29) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureImpl$3.onFailure(FutureImpl.java:153) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:75) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:230) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.tryFail(PromiseImpl.java:23) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.onFailure(PromiseImpl.java:54) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureImpl$ListenerArray.onFailure(FutureImpl.java:268) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureBase.emitFailure(FutureBase.java:75) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.FutureImpl.tryFail(FutureImpl.java:230) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.tryFail(PromiseImpl.java:23) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.onFailure(PromiseImpl.java:54) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.handle(PromiseImpl.java:43) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.future.PromiseImpl.handle(PromiseImpl.java:23) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.sqlclient.impl.command.CommandResponse.fire(CommandResponse.java:46) ~[vertx-sql-client-4.1.3.jar:4.1.3]
        at io.vertx.sqlclient.impl.SocketConnectionBase.handleMessage(SocketConnectionBase.java:279) ~[vertx-sql-client-4.1.3.jar:4.1.3]
        at io.vertx.sqlclient.impl.SocketConnectionBase.lambda$init$0(SocketConnectionBase.java:98) ~[vertx-sql-client-4.1.3.jar:4.1.3]
        at io.vertx.core.net.impl.NetSocketImpl.lambda$new$1(NetSocketImpl.java:97) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.streams.impl.InboundBuffer.handleEvent(InboundBuffer.java:240) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.streams.impl.InboundBuffer.write(InboundBuffer.java:130) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.net.impl.NetSocketImpl.lambda$handleMessage$9(NetSocketImpl.java:390) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:50) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.ContextImpl.emit(ContextImpl.java:274) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:22) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.net.impl.NetSocketImpl.handleMessage(NetSocketImpl.java:389) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.net.impl.ConnectionBase.read(ConnectionBase.java:155) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.vertx.core.net.impl.VertxHandler.channelRead(VertxHandler.java:154) ~[vertx-core-4.1.3.jar:4.1.3]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) ~[netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) ~[netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) ~[netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) ~[netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.vertx.db2client.impl.codec.DB2Encoder.lambda$write$0(DB2Encoder.java:72) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.DB2Decoder.decodePayload(DB2Decoder.java:96) [vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.DB2Decoder.decode(DB2Decoder.java:52) [vertx-db2-client-4.1.3.jar:4.1.3]
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:507) [netty-codec-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:446) [netty-codec-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-codec-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:719) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:655) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:581) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) [netty-transport-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) [netty-common-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-common-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-common-4.1.67.Final.jar:4.1.67.Final]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_302]
Caused by: java.lang.IndexOutOfBoundsException: readerIndex(32741) + length(17) exceeds writerIndex(32757): ReadOnlyByteBuf(ridx: 32741, widx: 32757, cap: 32757/32757, unwrapped: PooledSlicedByteBuf(ridx: 0, widx: 32757, cap: 32757/32757, unwrapped: PooledUnsafeDirectByteBuf(ridx: 32860, widx: 32860, cap: 65536)))
        at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1442) ~[netty-buffer-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.buffer.AbstractByteBuf.checkReadableBytes(AbstractByteBuf.java:1428) ~[netty-buffer-4.1.67.Final.jar:4.1.67.Final]
        at io.netty.buffer.AbstractByteBuf.skipBytes(AbstractByteBuf.java:971) ~[netty-buffer-4.1.67.Final.jar:4.1.67.Final]
        at io.vertx.db2client.impl.drda.Cursor.skipFdocaBytes(Cursor.java:1669) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.drda.Cursor.calculateColumnOffsetsForRow_(Cursor.java:1488) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.drda.Cursor.stepNext(Cursor.java:220) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.drda.Cursor.next(Cursor.java:232) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.RowResultDecoder.next(RowResultDecoder.java:52) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.ExtendedQueryCommandBaseCodec.decodePreparedQuery(ExtendedQueryCommandBaseCodec.java:81) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.ExtendedQueryCommandCodec.decodeQuery(ExtendedQueryCommandCodec.java:49) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.QueryCommandBaseCodec.decodePayload(QueryCommandBaseCodec.java:69) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        at io.vertx.db2client.impl.codec.DB2Decoder.decodePayload(DB2Decoder.java:79) ~[vertx-db2-client-4.1.3.jar:4.1.3]
        ... 21 more

gbadner avatar Sep 01 '21 20:09 gbadner

I've created a test on this branch: https://github.com/DavideD/vertx-sql-client/commit/d117691d3486f2ef2c4d7ec26670148a57b9c3a3

DavideD avatar Sep 21 '21 10:09 DavideD

Any update on this issue?

DavideD avatar Oct 20 '21 14:10 DavideD

@mswatosh ?

vietj avatar Oct 21 '21 06:10 vietj

can you provide a reproducer for SQL client ?

vietj avatar Oct 21 '21 06:10 vietj

I've created a test for QueryVariationsTest:

  @Test
  public void testSelectSysibmColumns(TestContext ctx) {
    connect(ctx.asyncAssertSuccess(conn -> conn
        .query("select table_name as TABLE_NAME, column_name as COLUMN_NAME, " +
                   "             case when data_type = 'CHARACTER VARYING' then 'VARCHAR' else data_type end as TYPE_NAME, " +
                   "             null as COLUMN_SIZE, is_nullable as IS_NULLABLE " +
                   "from sysibm.columns " +
                   "order by table_catalog, table_schema, table_name, column_name, ordinal_position" )
        .execute(
          ctx.asyncAssertSuccess(rowSet -> {
            ctx.assertEquals(1, rowSet.size());
            ctx.assertEquals(Arrays.asList("TABLE_NAME", "COLUMN_NAME", "COLUMN_SIZE", "IS_NULLABLE"), rowSet.columnsNames());
            RowIterator<Row> rows = rowSet.iterator();
            ctx.assertTrue(rows.hasNext());
            Row row = rows.next();
            // TODO: Add assertions about the content of the row
            ctx.assertFalse(rows.hasNext());
            conn.close();
          })
        )
    ));
  }

You should be able to cherry-pick this commit: https://github.com/DavideD/vertx-sql-client/commit/d117691d3486f2ef2c4d7ec26670148a57b9c3a3 (It's been a while)

DavideD avatar Oct 21 '21 07:10 DavideD

thanks a lot @DavideD I'll have a look

vietj avatar Oct 21 '21 13:10 vietj

@vietj I won't have time until monday to dig into this one, but let me know if you'd like me to take a look then. I took a quick scan but I didn't see an obvious quick fix.

mswatosh avatar Oct 21 '21 19:10 mswatosh

thanks, I'll have a look today as there is a reproducer, if I don't find I'll get back to you :-)

On Thu, Oct 21, 2021 at 9:57 PM Mark Swatosh @.***> wrote:

@vietj https://github.com/vietj I won't have time until monday to dig into this one, but let me know if you'd like me to take a look then. I took a quick scan but I didn't see an obvious quick fix.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eclipse-vertx/vertx-sql-client/issues/1033#issuecomment-948956555, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABXDCVIHHGA6WVW7LIXIVDUIBWA7ANCNFSM5DHLSAIQ .

vietj avatar Oct 25 '21 08:10 vietj

@mswatosh this seems to go beyond my knowledge of DRDA, can you handle it ?

vietj avatar Oct 25 '21 13:10 vietj

It looks like the issue here is that the response exceeds maximum length for a DSS response, so the driver needs to send a CNTQRY to get more data. Unfortunately, it looks like most of the path for that is not implemented, including having the cursor handle the partial row.

mswatosh avatar Nov 18 '21 20:11 mswatosh

is that something you can contribute @mswatosh ?

vietj avatar Nov 22 '21 15:11 vietj

@vietj I should be able to, it's just looking more like a small feature than the quick bug fix I was hoping for.

mswatosh avatar Nov 23 '21 16:11 mswatosh

is there anything you can do about it @mswatosh ?

vietj avatar Dec 06 '21 13:12 vietj

@vietj I've made some progress in my local environment, but I haven't had a lot of time and it's slow going since I have to learn how the code works as I go.

mswatosh avatar Dec 07 '21 16:12 mswatosh