Skip to content

pickle does not serialize Exception __cause__ field #73652

Description

@diekhans
mannequin
BPO 29466
Nosy @brettcannon, @ncoghlan, @diekhans, @avassalotti, @ericsnowcurrently, @serhiy-storchaka, @tjb900, @kakshay21, @iritkatriel
Files
  • cause_pickle.py: bug demog program
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2017-02-06.18:36:36.651>
    labels = ['type-bug', 'library', '3.11']
    title = 'pickle does not serialize Exception __cause__ field'
    updated_at = <Date 2022-01-06.16:21:58.240>
    user = 'https://github.com/diekhans'

    bugs.python.org fields:

    activity = <Date 2022-01-06.16:21:58.240>
    actor = 'iritkatriel'
    assignee = 'none'
    closed = False
    closed_date = None
    closer = None
    components = ['Library (Lib)']
    creation = <Date 2017-02-06.18:36:36.651>
    creator = 'diekhans'
    dependencies = []
    files = ['46550']
    hgrepos = []
    issue_num = 29466
    keywords = []
    message_count = 6.0
    messages = ['287163', '287304', '335686', '396603', '396604', '409871']
    nosy_count = 9.0
    nosy_names = ['brett.cannon', 'ncoghlan', 'diekhans', 'alexandre.vassalotti', 'eric.snow', 'serhiy.storchaka', 'tjb900', 'kakshay', 'iritkatriel']
    pr_nums = []
    priority = 'normal'
    resolution = None
    stage = None
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue29466'
    versions = ['Python 3.11']

    Activity

    1. diekhans commented on Feb 6, 2017

      diekhansmannequin
      MannequinAuthor

      python3 pickle does not serialize the __cause__ field, as shown by the attached demo program.

    2. added
      stdlibStandard Library Python modules in the Lib/ directory
      type-bugAn unexpected behavior, bug, or error
      on Feb 6, 2017
    3. serhiy-storchaka commented on Feb 8, 2017

      @serhiy-storchaka
      Member

      True. Attributes __context__, __cause__ and __traceback__ are not pickled. The traceback objects are even not pickleable.

      What is worse, some other non-special attributes are lost during pickling. For example name and path attributes of ImportError.

      >>> try: import foo
      ... except Exception as ex: exc = ex
      ... 
      >>> exc.name
      'foo'
      >>> exc.__reduce__()
      (<class 'ModuleNotFoundError'>, ("No module named 'foo'",), {})

      Or the value attribute of StopIteration if it was not passed to the constructor.

      >>> exc = StopIteration()
      >>> exc.value = 42
      >>> exc.__reduce__()
      (<class 'StopIteration'>, (), {})
    4. kakshay21 commented on Feb 16, 2019

      kakshay21mannequin
      Mannequin

      Hey, can I work on this?

    5. iritkatriel commented on Jun 27, 2021

      @iritkatriel
      Member

      I get different output for Serhiy's first example now, but the same for the second:

      >>> try: import foo
      ... except Exception as ex: exc = ex
      ... 
      >>> exc.name
      'foo'
      >>> exc.__reduce__()
      (<class 'ModuleNotFoundError'>, ("No module named 'foo'",), {'name': 'foo'})
      >>> exc = StopIteration()
      >>> exc.value = 42
      >>> exc.__reduce__()
      (<class 'StopIteration'>, ())
      >>>
    6. iritkatriel commented on Jun 27, 2021

      @iritkatriel
      Member
    7. iritkatriel commented on Jan 6, 2022

      @iritkatriel
      Member

      Attributes __context__, __cause__ and __traceback__ are not pickled. The traceback objects are even not pickleable.

      It's not guaranteed that the __cause__ and __context__ are picklable either.

      Should we try to pickle them, or should we document this and suggest traceback.TracebackException for storing or transmitting exception information?

    8. transferred this issue fromon Apr 10, 2022
    9. csm10495 commented on Apr 7, 2023

      @csm10495
      Contributor

      Some stuff may be lost in time here, though for my benefit: Why don't the fields on exceptions go into __dict__? If they were there wouldn't serialization work closer-to 'for free'?

    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

    Metadata

    Metadata

    Assignees

    No one assigned

      Labels

      3.11only security fixesstdlibStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or error

      Projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions