Cannot unpack data in java which is packed in Python

[Gajanana]

We are packing rest api response using python. contents are statuscode , actual response and response headers

def pack(self, obj: ProxyResponse) -> bytes: return msgpack.packb( [obj.status_code, obj.content, obj.headers], use_bin_type=True ) example 200 {"username":"[email protected]","provision_status":"PROVISIONED","account_type":"STANDALONE","region":"sa-east","application_id":"11208371-3f36-4baf-a2d3-cdd0cc087af5","platform_customer_id":"0a1da86ab79411ebbdb65ad800896ca7","application_instance_id":"631401d2-c55c-409b-8ec3-aa174c7d9b48","msp_id":null,"application_customer_id":"cc6c1db6b7c111eb924e5ad800896ca7"}

we are trying unpack it using java

If i get it as string its printed as below

my text �A�A�����w{"username":"[email protected]","provision_status / PROVISIONED account_typ@LSTANDAL reg 9 2sa-east applicat _id 11208371-3f36-4baf-a2d3-cdd0cc087af5 7platform_customer�=0a1da86ab79411ebbdb65ad800896ca7 9�qn �nce�<631401d2-c55c-409b-8ec3-aa174c7d 9b48","msp`/null�N��cc6c1 b7c1@�924e�� }��Content-T!-���/json�ccs-transact! -id�$52425ddb-3d1a-430a-9655-a4ca8f3f407 ���|Z|@� Byte array looks like below

com.abc.CCSCacheService my bytes [-17, -65, -67, -17, -65, -67, -17, -65, -67, -17, -65, -67, 1, 119, 123, 34, 117, 115, 101, 114, 110, 97, 109, 101, 34, 58, 34, 104, 99, 108, 111, 117, 100, 50, 48, 51, 43, 116, 114, 105, 116, 111, 110, 64, 103, 109, 97, 105, 108, 46, 99, 111, 109, 34, 44, 34, 112, 114, 111, 118, 105, 115, 105, 111, 110, 95, 115, 116, 97, 116, 117, 115, 34, 58, 34, 80, 82, 79, 86, 73, 83, 73, 79, 78, 69, 68, 34, 44, 34, 97, 99, 99, 111, 117, 110, 116, 95, 116, 121, 112, 101, 34, 58, 34, 83, 84, 65, 78, 68, 65, 76, 79, 78, 69, 34, 44, 34, 114, 101, 103, 105, 111, 110, 34, 58, 34, 115, 97, 45, 101, 97, 115, 116, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 95, 105, 100, 34, 58, 34, 49, 49, 50, 48, 56, 51, 55, 49, 45, 51, 102, 51, 54, 45, 52, 98, 97, 102, 45, 97, 50, 100, 51, 45, 99, 100, 100, 48, 99, 99, 48, 56, 55, 97, 102, 53, 34, 44, 34, 112, 108, 97, 116, 102, 111, 114, 109, 95, 99, 117, 115, 116, 111, 109, 101, 114, 95, 105, 100, 34, 58, 34, 48, 97, 49, 100, 97, 56, 54, 97, 98, 55, 57, 52, 49, 49, 101, 98, 98, 100, 98, 54, 53, 97, 100, 56, 48, 48, 56, 57, 54, 99, 97, 55, 34, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 95, 105, 110, 115, 116, 97, 110, 99, 101, 95, 105, 100, 34, 58, 34, 54, 51, 49, 52, 48, 49, 100, 50, 45, 99, 53, 53, 99, 45, 52, 48, 57, 98, 45, 56, 101, 99, 51, 45, 97, 97, 49, 55, 52, 99, 55, 100, 57, 98, 52, 56, 34, 44, 34, 109, 115, 112, 95, 105, 100, 34, 58, 110, 117, 108, 108, 44, 34, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 95, 99, 117, 115, 116, 111, 109, 101, 114, 95, 105, 100, 34, 58, 34, 99, 99, 54, 99, 49, 100, 98, 54, 98, 55, 99, 49, 49, 49, 101, 98, 57, 50, 52, 101, 53, 97, 100, 56, 48, 48, 56, 57, 54, 99, 97, 55, 34, 125, -17, -65, -67, -17, -65, -67, 67, 111, 110, 116, 101, 110, 116, 45, 84, 121, 112, 101, -17, -65, -67, 97, 112, 112, 108, 105, 99, 97, 116, 105, 111, 110, 47, 106, 115, 111, 110, -17, -65, -67, 99, 99, 115, 45, 116, 114, 97, 110, 115, 97, 99, 116, 105, 111, 110, 45, 105, 100, -17, -65, -67, 36, 53, 50, 52, 50, 53, 100, 100, 98, 45, 51, 100, 49, 97, 45, 52, 51, 48, 97, 45, 57, 54, 53, 53, 45, 97, 52, 102, 99, 97, 56, 102, 51, 102, 52, 48, 55, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

``

MessageUnpacker unpacker = MessagePack.newDefaultUnpacker(new ByteArrayInputStream(msgpack.getBytes(StandardCharsets.UTF_8)));

    log.warn("getPlatFormCustomers from redis {} {}", unpacker.getTotalReadBytes(),msgpack);
   try {
       int i = unpacker.unpackInt();
       log.warn("getPlatFormCustomers i {}",i);
       String s = unpacker.unpackString();
       log.warn("getPlatFormCustomers s {}",s);
       String s1 = unpacker.unpackString();
       log.warn("getPlatFormCustomers s1 {}",s1);
   }

``

But getting below exception

`` 2021-05-27T16:48:36,356 WARN [visualrf] Main com.abc.CCSCacheService unpacker3 excdeption org.msgpack.core.MessageTypeException: Expected String, but got Integer (32) |>> at o org.msgpack.core.MessageTypeException: Expected String, but got Integer (32) at org.msgpack.core.MessageUnpacker.unexpected(MessageUnpacker.java:595) ~[msgpack-core-0.8.22.jar:0.8.22] at org.msgpack.core.MessageUnpacker.unpackRawStringHeader(MessageUnpacker.java:1427) ~[msgpack-core-0.8.22.jar:0.8.22] at org.msgpack.core.MessageUnpacker.unpackString(MessageUnpacker.java:1140) ~[msgpack-core-0.8.22.jar:0.8.22]

``

also tried this

`` try{ MessageUnpacker unpacker2 = MessagePack.newDefaultUnpacker(msgpack.getBytes());

         int length = unpacker2.unpackBinaryHeader();
         ValueType valueType = unpacker2.getNextFormat().getValueType();
         byte[] unpackedArray = unpacker2.readPayload(length);
         Inflater decompresser = new Inflater();
         decompresser.setInput(unpackedArray);
         byte[] decompressedArray = new byte[length * 20];
         decompresser.inflate(decompressedArray);
         decompresser.end();
         log.warn("output is {}" ,new String(decompressedArray, "UTF-8"));

}

``

This is giving ``

gine.java:95) [vrf-engine-4.0.4.jar:?] |>> org.msgpack.core.MessageTypeException: Expected Binary, but got Integer (ef) at org.msgpack.core.MessageUnpacker.unexpected(MessageUnpacker.java:595) ~[msgpack-core-0.8.22.jar:0.8.22] at org.msgpack.core.MessageUnpacker.unpackBinaryHeader(MessageUnpacker.java:1464) ~[msgpack-core-0.8.22.jar:0.8.22] ``

Can somebody help in unpacking this data

[Gajanana]

@komamitsu @xerial . Can somebody help on this or point me to the documentation for this

[komamitsu]

Sorry for delayed reply.

MessageUnpacker unpacker = MessagePack.newDefaultUnpacker(new ByteArrayInputStream(msgpack.getBytes(StandardCharsets.UTF_8)));

This looks you handled the serialized MessagePack data as a String. It should be treated as a byte array.

BTW, the question and example code seem complicated. It would be great if you give a minimum reproducible code.