bitmex example not working

[ghost]

no modifications, checked out this project, running bitmex example at xchange-stream/xchange-bitmex/src/test/java/info/bitrich/xchangestream/bitmex/BitmexManualExample.java

Exception in thread "main" io.netty.handler.codec.CodecException: invalid WebSocket Extension handshake for "permessage-deflate; client_no_context_takeover"

Is there something i do wrong?

[jkolobok]

+1

[thebubbleindex]

Please see my comment on what I found in the hopes of fixing this issue here: https://github.com/bitrich-info/xchange-stream/pull/96

[jkolobok]

@thebubbleindex does the example work for you now? I added

    @Override
    protected WebSocketClientExtensionHandler getWebSocketClientExtensionHandler() {
        return info.bitrich.xchangestream.bitmex.netty.WebSocketClientCompressionAllowClientNoContextHandler.INSTANCE;
    }

where WebSocketClientCompressionAllowClientNoContextHandler is copy&pasted from GDAX. Now it gets connected without that exception, gets welcome message fromthe exchange. Then I try to subscribe to book updates and (according to websocks proxy I use) I get success message, but it's not able to decode it and throws this:

io.netty.handler.codec.compression.DecompressionException: decompression failure
	at io.netty.handler.codec.compression.JdkZlibDecoder.decode(JdkZlibDecoder.java:230) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1414) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:945) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.embedded.EmbeddedChannel.writeInbound(EmbeddedChannel.java:325) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.http.websocketx.extensions.compression.DeflateDecoder.decode(DeflateDecoder.java:68) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.http.websocketx.extensions.compression.PerMessageDeflateDecoder.decode(PerMessageDeflateDecoder.java:64) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.http.websocketx.extensions.compression.PerMessageDeflateDecoder.decode(PerMessageDeflateDecoder.java:30) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1388) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1159) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1202) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:489) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:428) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:265) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.handler.proxy.ProxyHandler.channelRead(ProxyHandler.java:255) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1414) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:945) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:141) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.21.Final.jar:4.1.21.Final]
	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]
Caused by: java.util.zip.DataFormatException: invalid distance too far back
	at java.util.zip.Inflater.inflateBytes(Native Method) ~[?:1.8.0_171]
	at java.util.zip.Inflater.inflate(Inflater.java:259) ~[?:1.8.0_171]
	at io.netty.handler.codec.compression.JdkZlibDecoder.decode(JdkZlibDecoder.java:187) ~[netty-all-4.1.21.Final.jar:4.1.21.Final]
	... 48 more

This is how communication looks like in CharlesProxy

[jkolobok]

Actually

 @Override
    protected WebSocketClientExtensionHandler getWebSocketClientExtensionHandler() {
        return null;
    }

works better

[thebubbleindex]

Yes, basically that is what I had to do as well, i.e. WebSocketClientExtensionHandler clientExtensionHandler = null; in NettyStreamingService.

I am not sure I know enough about the architecture of this project and NettyStreamingService to say what exactly this handler is doing and whether this is the root cause.

[jkolobok]

so here I cerated a pull request that makes bitmex somewhat working. I'm still testing it and improving, but at least it can connect and process books/trades and send orders