Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 554: Issues with data passing and pickle #3309

Open
jima80525 opened this issue Aug 28, 2023 · 1 comment
Open

PEP 554: Issues with data passing and pickle #3309

jima80525 opened this issue Aug 28, 2023 · 1 comment

Comments

@jima80525
Copy link

jima80525 commented Aug 28, 2023

I believe the example program given in PEP 554: https://peps.python.org/pep-0554/#passing-objects-via-pickle has some issues. It's based on a zero value (b'\x00') being a data terminator, but my attempts to recreate this (using threads instead of interpreters) showed that pickle will insert zero values in the middle of the data stream:

Python 3.10.4 (main, May  7 2022, 16:23:10) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pickle
>>> pickle.dumps("PEP 554")
b'\x80\x04\x95\x0b\x00\x00\x00\x00\x00\x00\x00\x8c\x07PEP 554\x94.'

I think the goal here is to simply demonstrate passing pickled objects through the pipe. To that end, it might make a better example to change the reading portion to read a fixed, relatively large, buffer at once and then decode that. Something like:

    data = os.read(reader, 1024)
    obj = pickle.loads(data)
    do_something(obj)

This is clearly not a general purpose solution, but I think for an example like this, not solving all of the corner cases is OK.

Thanks for all of the tremendous work already put into this! This will be very cool!

@Rosuav
Copy link
Contributor

Rosuav commented Aug 28, 2023

@ericsnowcurrently Your view on this?

If I were to suggest a colour for this particular bike shed, it would be a length-preceded format.

@hugovk hugovk changed the title PEP 554 Issues with data passing and pickle PEP 554: Issues with data passing and pickle Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants
@Rosuav @jima80525 and others