When, why and how to use `valueOf()`?

[mattfysh]

I'm using pythonia, and my library code makes use of a handful of callbacks that pass arbitrary data between them. To get pythonia to work, I've had to use this code in a few places:

const ret = await python_function()
try {
  return await ret.valueOf()
} catch(e) {
  return ret
}

and on the flipside:

orig_cb = js_obj['callback']
def cb(x):
  ret = orig_cb(x)
  try:
    return ret.valueOf()
  except:
    return ret
js_obj['callback'] = cb

I was hoping to grok why it works this way, and if there's a better solution I should be using?

Also I am passing a nodejs Buffer through to a python function, but it's coming through as a dict with keys type and data - so the python code needs to buf = bytes(buf['data']), is that something that should also be deserialized to bytes by python before invoking the function?

Cheers!

[extremeheat]

Buffer in JS is just a special class around Uint8Array, so when sent to Python it will be wrapped around in a JSProxy object (primitive data like strings or numbers are copied when sent between JS/Python, while objects are wrapped around a Proxy for interop). Doing a .valueOf() call will remove the proxy wrapper and serialize it to something that exists solely on the Python side.

This is explained in https://github.com/extremeheat/JSPyBridge#details.

So yes, this is intended behavior to allow interop, as inside the JS Buffer instance there is a type and a data field, the data field containing the array of bytes.

[mara004]

Hello, I'm trying to transfer a large bitmap buffer from JS to Python using buffer.valueOf()["data"] as suggested above. However, this seems to be really slow. Most of the time is not spent rendering, but transferring the data. Is this expected? Any ideas how to improve performance? Here's my use case: https://gist.github.com/mara004/87276da4f8be31c80c38036c6ab667d7

[mara004]

Not sure, but from what I can guess, it looks like data transfer is done by serializing the Uint8Array to json, writing the result into a pipe, and reading/deserializing it in python? If that's true, the performance issue would be explained.

It should be much better if the array's memory could be directly passed as binary data in some dedicated channel, avoiding the json dump/load layer and the inefficient representation (an array as ascii json string needs 3 to 5 times the memory compared to just the array's actual byte data).

As a workaround, I guess I could do this myself already by writing the data into some socket or tempfile from JS, though...

[mara004]

Indeed using a tempfile already results in a significant speedup.

[mara004]

@extremeheat I took the time to dive into the code and create a patch (#103) that would fix the problem described above with a new blobValueOf() function to transfer binary data from JS to Python without JSON serialization.