dubbo icon indicating copy to clipboard operation
dubbo copied to clipboard

Triple attachment does not support non-ASCII characters

Open xlq20080808 opened this issue 1 year ago • 3 comments

  • [x] I have searched the issues of this repository and believe that this is not a duplicate.

Environment

  • Dubbo version: 3.2.5
  • Java version: 1.8

Steps to reproduce this issue

  1. Insert a value containing non-ASCII characters in the attachment
  2. Get the attachment in the provider, where non-ascii characters are converted to "?"

reproduce this issue.

import org.apache.dubbo.rpc.Filter;
import org.apache.dubbo.rpc.Invocation;
import org.apache.dubbo.rpc.Invoker;
import org.apache.dubbo.rpc.Result;
import org.apache.dubbo.rpc.RpcContext;
import org.apache.dubbo.rpc.RpcException;

public class ClientContextAppender implements Filter {
    public ClientContextAppender() {
    }

    public Result invoke(Invoker<?> invoker, Invocation invocation) throws RpcException {
        RpcContext context = RpcContext.getClientAttachment();
        context.setAttachment("non-Ascii", "zh这是中文zh");
        return invoker.invoke(invocation);
    }
}
image

Expected Behavior

Consistent with the attachment of the dubbo protocol, The triple protocol can also pass values containing non-ASCII characters

Actual Behavior

Non-ASCII characters are converted to '?'

Problem Code

The netty's huffman encoding condition image

Not hit netty's huffman encoding condition

When netty does not hit the huffman encoding of the header, it will convert the non-ascii code to '?'

c2b:1857, AsciiString (io.netty.util)
writeAscii:1188, ByteBufUtil (io.netty.buffer)
setCharSequence0:717, AbstractByteBuf (io.netty.buffer)
writeCharSequence:1187, AbstractByteBuf (io.netty.buffer)
encodeStringLiteral:279, HpackEncoder (io.netty.handler.codec.http2)
encodeLiteral:306, HpackEncoder (io.netty.handler.codec.http2)
encodeHeader:194, HpackEncoder (io.netty.handler.codec.http2)
encodeHeadersIgnoreMaxHeaderListSize:145, HpackEncoder (io.netty.handler.codec.http2)
encodeHeadersEnforceMaxHeaderListSize:137, HpackEncoder (io.netty.handler.codec.http2)
encodeHeaders:118, HpackEncoder (io.netty.handler.codec.http2)
encodeHeaders:74, DefaultHttp2HeadersEncoder (io.netty.handler.codec.http2)
...
image

Hit netty's huffman encoding condition

4.1.86.Final Netty has a bug in the conversion

When netty hits the huffman encoding condition of the header, non-ASCII characters are cleaned to the ASCII range image Such a convert may cause unexpected errors Non-ASCII characters may be converted to characters less than 32 For example, envoy will check the value of the header, and if it finds characters less than 32, it will report an error

xlq20080808 avatar Aug 15 '23 09:08 xlq20080808

Base64 encoding and decoding of non-ASCII code attachment can solve this problem

xlq20080808 avatar Aug 15 '23 10:08 xlq20080808

Because the header of HTTP2 does not support non-ASCII code.

EarthChen avatar Aug 16 '23 07:08 EarthChen

Because the header of HTTP2 does not support non-ASCII code.

This is the case, which will cause the attachment to different behavior under the dubbo and triple protocols. Is it necessary for us to support it? I tried to use base64 encode to achieve it #12905

xlq20080808 avatar Aug 16 '23 08:08 xlq20080808