Enable PandasCodec.decode_request to restore the exact dataframe

[ysk24ok]

trafficstars

What

When a dataframe is encoded by PandasCodec.encode_request(use_bytes=True), PandasCodec.decode_request() cannot restore the exact dataframe.

client code

X = pd.DataFrame(
    dict(
        int_col=[1, 2, 3],
        str_col=["s1", "s2", "s3"],
    )
)
transport = "rest"

if transport == "rest"
    # `use_bytes` must be False
    # because `request.post` throws `TypeError: Object of type bytes is not JSON serializable`
    request = PandasCodec.encode_request(X, use_bytes=False)
    endpoint = f"http://localhost:8080/v2/models/{MODEL_NAME}/infer"
    response = requests.post(endpoint, json=request.dict())
    print(response.json())
else:
    with grpc.insecure_channel("localhost:8081") as channel:
        stub = dataplane_pb2_grpc.GRPCInferenceServiceStub(channel)
        # `use_bytes` must be True
        # because `ModelInferRequestConverter.from_types` throws `TypeError: 's1' has type str, but expected one of: bytes`
        request = PandasCodec.encode_request(X)
        model_infer_request = converters.ModelInferRequestConverter.from_types(request, model_name="test")
        model_infer_response = stub.ModelInfer(model_infer_request)
        response = converters.ModelInferResponseConverter.to_types(model_infer_response)
        print(response.json())

server code

class TestModel(MLModel):
    async def predict(self, request: InferenceRequest) -> InferenceResponse:
        X_df = PandasCodec.decode_request(request)
        print(X_df)
        return ...

server log (rest)

   int_col str_col
0        1      s1
1        2      s2
2        3      s3

server log (grpc)

This is different from the original df.

   int_col str_col
0        1   b's1'
1        2   b's2'
2        3   b's3'

Solution

I think the easiest way to add use_bytes or equivalent arg to PandasCodec.decode_request but it's kind of hassle to change the value depending on the use_bytes value on the client (PandasCodec.encode_request) side. It'd be ideal if we could add the use_bytes value to the request so it can be used on the server side, but not sure if it's possible.

Or is there any way to avoid this issue?