PEP 554: Issues with data passing and pickle

[jima80525]

I believe the example program given in PEP 554: https://peps.python.org/pep-0554/#passing-objects-via-pickle has some issues. It's based on a zero value (b'\x00') being a data terminator, but my attempts to recreate this (using threads instead of interpreters) showed that pickle will insert zero values in the middle of the data stream:

Python 3.10.4 (main, May  7 2022, 16:23:10) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> pickle.dumps("PEP 554")
b'\x80\x04\x95\x0b\x00\x00\x00\x00\x00\x00\x00\x8c\x07PEP 554\x94.'

I think the goal here is to simply demonstrate passing pickled objects through the pipe. To that end, it might make a better example to change the reading portion to read a fixed, relatively large, buffer at once and then decode that. Something like:

    data = os.read(reader, 1024)
    obj = pickle.loads(data)
    do_something(obj)

This is clearly not a general purpose solution, but I think for an example like this, not solving all of the corner cases is OK.

Thanks for all of the tremendous work already put into this! This will be very cool!

[Rosuav]

@ericsnowcurrently Your view on this?

If I were to suggest a colour for this particular bike shed, it would be a length-preceded format.