Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Undefined objects can't be copied or pickled by Python > 3.5 #2025

Open
nitzmahone opened this issue Oct 1, 2024 · 2 comments · May be fixed by #2026
Open

Undefined objects can't be copied or pickled by Python > 3.5 #2025

nitzmahone opened this issue Oct 1, 2024 · 2 comments · May be fixed by #2026

Comments

@nitzmahone
Copy link

nitzmahone commented Oct 1, 2024

Description

Due to an ancient erroneous deletion of __slots__ metadata, it's not possible to copy/deepcopy/pickle Undefined objects (all of which we occasionally need to do in Ansible) in Python > 3.5.

The comment above the deletion of __slots__ implies a misunderstanding of how that metadata was/is used. The deletion was always probably technically a bug, but once Python started providing default implementations of __getstate__ and __reduce_ex__ on all objects (which consult that metadata at runtime), it broke the default copyability/pickle-ability of Undefined (and derived) objects.

We're working around this with a monkeypatch that restores the missing __slots__ metadata. We've also submitted PR #2026 to zap the problematic deletion and tests that verify copy/deepcopy/pickle functionality on all the built-in Undefined types.

Repro

(on any Python > 3.5 with Jinja installed):

$ python -c 'import copy; import jinja2; copy.copy(jinja2.Undefined())'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib64/python3.11/copy.py", line 92, in copy
    rv = reductor(4)
         ^^^^^^^^^^^
TypeError: cannot pickle 'Undefined' object

Expected

The default runtime-provided implementations of __getstate__ and __reduce_ex__ that back copy.copy, copy.deepcopy and pickle should work properly with all Undefined derived types.

Environment

  • Python version: >= 3.6
  • Jinja version: anything from at least the past 16 years
@nitzmahone nitzmahone linked a pull request Oct 1, 2024 that will close this issue
@davidism
Copy link
Member

davidism commented Oct 1, 2024

I've been confused by that code as well. If the comment isn't accurate and everything still works, I'm fine with a PR for that.

@nitzmahone
Copy link
Author

Cool- there's another small related issue + fix inbound that we tripped over while writing the tests for this one (my colleague is filing a separate issue + PR right now).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants