Skip to content

Unable to inspect class with cyfunction __init__ due to unwrap and descriptor behavior changed #137317

Closed

Description

@yanyongyu

Bug report

Bug description:

Background:

Pydantic v1 modified builtin dataclass's __init__ method (wraps). When inspect on the modified dataclass, the class's signature will contain a self parameter.

Reproduce code with pydantic v1 installed:

from dataclasses import dataclass
import inspect

from pydantic import BaseConfig
from pydantic.dataclasses import _add_pydantic_validation_attributes

@dataclass
class A:
    x: int

print(inspect.signature(A).parameters)  # x

_add_pydantic_validation_attributes(A, BaseConfig, False, "")

print(inspect.signature(A).parameters)  # self, x

print(A.__init__, hasattr(A.__init__, "__wrapped__"))  # cyfunction A.__init__, True
meth = inspect._descriptor_get(A.__init__, A)
print(meth, hasattr(A.__init__, "__wrapped__"))  # bound method A.__init__, True
meth = inspect.unwrap(A.__init__, stop=lambda m: hasattr(m, "__signature__"))
print(meth)  # function A.__init__, this cause the signature error

This error is because of the unwrap and descriptor get behavior at

cpython/Lib/inspect.py

Lines 1924 to 1926 in fe0e921

meth = _descriptor_get(meth, cls)
if follow_wrapper_chains:
meth = unwrap(meth, stop=lambda m: hasattr(m, "__signature__"))

The cyfunction __init__ descriptor return a bound method dataclass.__init__ and then unwrap it back to a function dataclass.__init__.

Related to #132055

CPython versions tested on:

3.13

Operating systems tested on:

Linux

Linked PRs

Activity

  1. StanFromIreland commented on Aug 2, 2025

    @StanFromIreland
    Member

    I cannot reproduce with Pydantic 1.10.0 and 3.13.5.

  2. yanyongyu commented on Aug 2, 2025

    @yanyongyu
    ContributorAuthor

    i'm using pydantic 1.10.22 and find this issue.

  3. MingxuanGame commented on Aug 2, 2025

    @MingxuanGame

    reproduced with pydantic>=1.10.20,<2

  4. ZeroIntensity commented on Aug 2, 2025

    @ZeroIntensity
    Member

    I couldn't reproduce this with pydantic==1.10.20 on 3.13.5 or 3.12.10. But anyway, this looks like the cost of using private APIs; by using inspect._descriptor_get (and _add_pydantic_validation_attributes, probably), you opt out of backwards compatibility.

    Edit: Ah, wait, I do see a difference in behavior on 3.13.5. I was using 3.13.3. But my point still stands about private APIs; you can't rely on implementation details and expect it not to break.

  5. yanyongyu commented on Aug 2, 2025

    @yanyongyu
    ContributorAuthor

    hi @ZeroIntensity, i known this is a private api of pydantic, but actually pydantic will modify standard dataclass and wrap it when passed to pydantic's verification using this way in v1.

    The code is just for reproduce. The production code is using pydantic public validation api.

    And, about inspect, we are using inspect to get the dataclass's signature. inspect.signature will call the _descriptor_get and unwrap when getting the class's __init__.

  6. ZeroIntensity commented on Aug 2, 2025

    @ZeroIntensity
    Member

    Ok, would you mind posting an example of how this affects Pydantic's public validation API? I suspect they're relying on implementation details under the hood, but it'd give us a clearer idea of the impact. It'd also be nice if you could point me to a Pydantic issue so I can communicate there. There might also be a Cython issue considering this seems to be with cyfunction.

  7. yanyongyu commented on Aug 2, 2025

    @yanyongyu
    ContributorAuthor

    I' will give an example to show the actual issue and also make a try whether issue exists without pydantic.

    Currently this issue is not posted to pydantic because i think the behavior changed only due to inspect (the related pr show in the issue). Our matrix unit tests show that this only exists when cpython 3.13 and pydantic v1, other test versions of stable cpython and pydantic v2 are ok.

  8. ProgramRipper commented on Aug 2, 2025

    @ProgramRipper

    We notice that cythonize may affect the result (Python 3.13.4, pydantic 1.10.22):

    pydantic.dataclasses=<module 'pydantic.dataclasses' from '.../.venv/lib/python3.13/site-packages/pydantic/dataclasses.py'>
    inspect.signature(A).parameters=mappingproxy(OrderedDict({'x': <Parameter "x: int">}))
    
    pydantic.dataclasses=<module 'pydantic.dataclasses' from '.../.venv/lib/python3.13/site-packages/pydantic/dataclasses.cpython-313-x86_64-linux-gnu.so'>
    inspect.signature(A).parameters=mappingproxy(OrderedDict({'self': <Parameter "self">, 'x': <Parameter "x: int">}))
    
  9. ProgramRipper commented on Aug 2, 2025

    @ProgramRipper

    I think the following code could indicate that #132055 introduce some strange behavior with cyfunction:

    import inspect
    from functools import wraps
    
    from pydantic import validator
    
    
    class A:
        def __init__(self):
            pass
    
    
    print(f"{A.__init__=}", f"{inspect.signature(A).parameters=}\n")
    
    print(f"set A.__init__ to a random wraped cyfunction: {validator=}")
    setattr(A, "__init__", wraps(A.__init__)(validator))
    
    print(f"{A.__init__=}", f"{inspect.signature(A).parameters=}")
    A.__init__=<function A.__init__ at 0x7f1554185120> inspect.signature(A).parameters=mappingproxy(OrderedDict())
    
    set A.__init__ to a random wraped cyfunction: validator=<cyfunction A.__init__ at 0x7f15541320c0>
    A.__init__=<cyfunction A.__init__ at 0x7f15541320c0> inspect.signature(A).parameters=mappingproxy(OrderedDict({'self': <Parameter "self">}))
    
  10. StanFromIreland commented on Aug 2, 2025

    @StanFromIreland
    Member

    Is it possible to get a pure CPython reproducer?

  11. ZeroIntensity commented on Aug 2, 2025

    @ZeroIntensity
    Member

    Yeah, this looks like a bug with Cython. Cython mimicks CPython, so there's a chance they just need to update things on their end to fix inspect.signature usage.

  12. 6 remaining items

  13. ZeroIntensity commented on Aug 3, 2025

    @ZeroIntensity
    Member

    I guess we can accept this as a bug. But, this is definitely in a gray area on what counts as "the correct behavior". Manually overriding __init__ with your own descriptor is really not something that inspect.signature was designed to handle.

    If you can come up with a fix that's simple and doesn't break anything else, we'd be happy to accept it.

  14. yanyongyu commented on Aug 4, 2025

    @yanyongyu
    ContributorAuthor

    I'm not sure why the second time of unwrap is added after the _descriptor_get. If you have any relevant information please let me know.

    I tested the case from #119605 (comment). If i remove the second time of unwrap, the case is still right and my case is ok too.


    Edit: The return value of _signature_get_user_defined_method is passed to _get_signature_of which also respect the follow_wrapper_chains and whether the input is a method (this will remove self or cls). Also, semantically, I think function _signature_get_user_defined_method should return a bound method instead of a function.

    This unwrap is added in commit 44f103f . It seems this commit is deal with a classmethod/staticmethod with function that wraps another. I tried to write some cases:

    Case 1: staticmethod with wrapped function

    from functools import wraps
    import inspect
    
    def modify_func(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            return func(*args, **kwargs)
    
        return wrapper
    
    def fake_init(x): ...
    
    class Foo:
        __init__ = staticmethod(modify_func(fake_init))
    
    print(inspect.signature(Foo).parameters)  # x

    In this case, _descriptor_get return the function modify_func.<locals>.wrapper and then unwrap into function fake_init. Then inspect get the signature of fake_init.

    If the unwrap is removed, inspect getting the signature of function modify_func.<locals>.wrapper also respect the follow_wrapper_chains and unwrap it. The result not changes.

    Case 2: classmethod with wrapped function

    from functools import wraps
    import inspect
    
    def modify_func(func):
        @wraps(func)
        def wrapper(*args, **kwargs):
            return func(*args, **kwargs)
    
        return wrapper
    
    def fake_init(cls, x): ...
    
    class Foo:
        __init__ = classmethod(modify_func(fake_init))
    
    print(inspect.signature(Foo).parameters)  # cls, x

    In this case, inspect returns the signature including cls parameter. It seems not right (cpython 3.12 returns x param only). _descriptor_get return the bound method of function modify_func.<locals>.wrapper and then unwrap into function fake_init. Then inspect get the signature of fake_init, and the cls param is also included here.

    If the unwrap is removed, inspect gets the signature of the bound method of function modify_func.<locals>.wrapper, which does not include the cls param. The result is x only.

    If there's any other case i don't cover, please let me know 😄 . I'm also happy to create a PR to fix this.

  15. yanyongyu commented on Aug 18, 2025

    @yanyongyu
    ContributorAuthor

    After supplementing with more test cases (follow the cases used in test_signature_on_class_with_new), it was found that there were also incorrect results when using classmethod or other method type with the wrapped __new__ function.

    https://github.com/yanyongyu/cpython/blob/9541a02832a5c0a4b16752a7893bac59b501d897/Lib/test/test_inspect/test_inspect.py#L4132-L4236

  16. yanyongyu commented on Aug 27, 2025

    @yanyongyu
    ContributorAuthor

    Hi @ZeroIntensity , I've tried to fix this issue in PR #137862 and added more test cases. Now it is ready for review.

  17. added a commit that references this issue on Aug 28, 2025
  18. added 2 commits that reference this issue on Aug 28, 2025
  19. serhiy-storchaka commented on Aug 28, 2025

    @serhiy-storchaka
    Member

    Thank you for your contribution @yanyongyu.

  20. added 2 commits that reference this issue on Aug 28, 2025
  21. added a commit that references this issue on Sep 9, 2025
  22. added a commit that references this issue on Oct 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixes3.15bugs and security fixesstdlibStandard Library Python modules in the Lib/ directorytopic-dataclassestype-bugAn unexpected behavior, bug, or error

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions