Kernel websocket messages should convert buffers to bytes objects

[nicholaswold]

https://github.com/jupyter-server/jupyter_server/blob/master/jupyter_server/services/kernels/handlers.py#L412

When handling a kernel message, Jupyter Server will currently fail to process any message that uses buffers because session.send expects the contents to be either bytes or bytearray, and JSON has no native way to represent either type. Jupyter Server should probably do some conversion here to ensure that kernel messages that utilize buffers are consumable by session.send.

I've solved the problem locally by injecting this code before session.send:

buffers = message.get("buffers")
message["buffers"] = [bytes(buffers)]

But I'm unsure if this is the correct solution. For example, I don't know if the messaging spec has an opinion about whether the buffers attribute is typed as List[bytes] or List[Union[bytes, List[bytes]]].

[kevin-bates]

Hi @nicholaswold - thanks for raising this issue.

Could you please share how this can be reproduced?

Also, might the suggested change be better located just following the json.loads() since the if portion of that block already deals with bytes? Perhaps your scenario only deals with JSON.

@Carreau or @SylvainCorlay (or anyone else familiar with the messaging protocol) are you able to comment on Nick's question regarding the expected type of the buffers attribute?

From what I can gather looking at the code and reading the messaging protocol, it seems like the type would be List[bytes] where each entry is a "buffer".

[nicholaswold]

I think you're right that this is a JSON-specific issue. I was replicated this by using the FileUpload widget on Nteract, which uses rxJS to serialize kernel messages into a JSON string, whereas Jupyter Notebook sends a binary message and doesn't seem to hit this problem.