unittest.mock does not understand dataclasses #80761

Activity

1. parejkoj commented on Apr 9, 2019

   

   parejkojmannequin

   on Apr 9, 2019

   MannequinAuthor

   More actions

   The new dataclasses.dataclass is very useful for describing the properties of a class, but it appears that Mocks of such decorated classes do not catch the members that are defined in the dataclass. I believe the root cause of this is the fact that unittest.mock.Mock generates the attributes of its spec object via dir, and the non-defaulted dataclass attributes do not appear in dir.

   Given the utility in building classes with dataclass, it would be very useful if Mocks could see the class attributes of the dataclass.

   Example code:

   import dataclasses
   import unittest.mock
   
   @dataclasses.dataclass
   class Foo:
       name: str
       baz: float
       bar: int = 12
   
   FooMock = unittest.mock.Mock(Foo)
   fooMock = FooMock()  # should fail: Foo.__init__ takes two arguments
   # I would expect these to be True, but they are False
   'name' in dir(fooMock)
   'baz' in dir(fooMock)
   'bar' in dir(fooMock)
2. parejkoj

   added

   3.7 (EOL)end of lifeend of life

   stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory

   testsTests in the Lib/test dirTests in the Lib/test dir

   type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

   on Apr 9, 2019
3. ericvsmith commented on Apr 9, 2019

   

   ericvsmith

   on Apr 9, 2019

   Member

   More actions

   I'm not sure why dataclasses would be different here:

   >>> import dataclasses
   >>> import unittest.mock
   >>> @dataclasses.dataclass
   ... class Foo:
   ...     name: str
   ...     baz: float
   ...     bar: int = 12
   ...
   >>> import inspect
   >>> inspect.signature(Foo)
   <Signature (name: str, baz: float, bar: int = 12) -> None>
   >>>

   Foo is just a normal class with a normal __init__.

   This is no different than if you don't use dataclasses:

   >>> class Bar:
   ...     def __init__(self, name: str, baz: float, bar: int = 12) -> None:
   ...         pass
   ...
   >>> Bar()
   Traceback (most recent call last):
     File "<stdin>", line 1, in <module>
   TypeError: __init__() missing 2 required positional arguments: 'name' and 'baz'
   >>> inspect.signature(Bar)
   <Signature (name: str, baz: float, bar: int = 12) -> None>
   >>> BarMock = unittest.mock.Mock(Bar)
   >>> barMock = BarMock()
4. tirkarthi commented on Apr 9, 2019

   

   tirkarthi

   on Apr 9, 2019

   Member

   More actions

   mock.Mock doesn't do signature validation by default for constructor and methods. This is expected. create_autospec [0] should be used to make sure the signature is validated.'

   import dataclasses
   import unittest.mock
   
   @dataclasses.dataclass
   class Foo:
       name: str
       baz: float
       bar: int = 12
   
   FooMock = unittest.mock.create_autospec(Foo)
   fooMock = FooMock()  # Will fail now since it's specced

   ➜  cpython git:(master) ./python.exe ../backups/bpo36580.py
   Traceback (most recent call last):
     File "../backups/bpo36580.py", line 11, in <module>
       fooMock = FooMock()  # should fail: Foo.__init__ takes two arguments
     File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 984, in __call__
       _mock_self._mock_check_sig(*args, **kwargs)
     File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/unittest/mock.py", line 103, in checksig
       sig.bind(*args, **kwargs)
     File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 3021, in bind
       return args[0]._bind(args[1:], kwargs)
     File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/inspect.py", line 2936, in _bind
       raise TypeError(msg) from None
   TypeError: missing a required argument: 'name'

   On the other hand 'name' in dir(FooMock) doesn't have the attributes (name and baz) present I suppose they are constructed dynamically when an object is created from Foo since they are present in dir(Foo()) and mock is not able to detect them? mock.create_autospec does an initial pass of dir(Foo) to copy the attributes [1] and perhaps it's not able to copy name and bar while baz is copied. Below are for FooMock = create_autospec(Foo) . So 'name' in dir(Foo) is False for dataclasses. Is this a known behavior?

   dir(Foo)

   ['__annotations__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'bar']

   dir(Foo(1, 2))

   ['__annotations__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'bar', 'baz', 'name']

   dir(create_autospec(Foo))

   ['__annotations__', '__class__', '__dataclass_fields__', '__dataclass_params__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'assert_any_call', 'assert_called', 'assert_called_once', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'assert_not_called', 'attach_mock', 'bar', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'method_calls', 'mock_add_spec', 'mock_calls', 'reset_mock', 'return_value', 'side_effect']

   print('name' in dir(fooMock)) # False
   print('baz' in dir(fooMock)) # False
   print('bar' in dir(fooMock)) # True

   [0] https://docs.python.org/3/library/unittest.mock.html#unittest.mock.create_autospec
   [1]

   cpython/Lib/unittest/mock.py

   Line 2263 in 0e10766

   for entry in dir(spec):
5. tirkarthi commented on Apr 9, 2019

   

   tirkarthi

   on Apr 9, 2019

   Member

   More actions

   To add to this mock.Mock also copies dir(spec) but creating an instance from mock doesn't copy it where it's not a problem with create_autospec. Mock with spec does only attribute validation whereas create_autospec does signature validation. There is another open issue to make mock use spec passed as if it's autospecced bpo-30587 where this could be used as a data point to change API. I am adding mock devs for confirmation.
6. tirkarthi

   added

   3.8 (EOL)end of lifeend of life

   on Apr 9, 2019
7. tirkarthi commented on Apr 11, 2019

   

   tirkarthi

   on Apr 11, 2019

   Member

   More actions

   Below is a even more simpler reproducer without dataclasses. 'name' is not listed as a class attribute in dir(Person) since it's not defined with a value but 'age' is with zero. Python seems to not to list something not defined with a value in declaration as a class attribute in dir(). Hence 'name' is not copied when Person is used as spec. spec only does attribute access validation. autospeccing [0] can be used for signature validation. The fields for dataclasses are defined in __dataclass_fields__ but I am not sure of special casing copying __dataclass_fields__ fields along with dir for dataclasses when normal Python doesn't list them as class attributes. If needed I would like dir(dataclass) to be changed to include __dataclass_fields__. I would propose closing as not a bug.

   # ../backups/dataclass_dir.py

   from unittest.mock import Mock
   
   class Person:
       name: str
       age: int = 0
   
       def foo(self):
           pass
   
   person_mock = Mock(spec=Person)
   print(dir(Person))
   print(dir(person_mock))
   person_mock.foo
   print("age" in dir(person_mock))
   print("name" in dir(person_mock))

   $ cpython git:(master) ./python.exe ../backups/dataclass_dir.py
   ['__annotations__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'age', 'foo']

   ['__annotations__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', 'age', 'assert_any_call', 'assert_called', 'assert_called_once', 'assert_called_once_with', 'assert_called_with', 'assert_has_calls', 'assert_not_called', 'attach_mock', 'call_args', 'call_args_list', 'call_count', 'called', 'configure_mock', 'foo', 'method_calls', 'mock_add_spec', 'mock_calls', 'reset_mock', 'return_value', 'side_effect']
   True
   False

   [0] https://docs.python.org/3/library/unittest.mock.html#autospeccing
8. tirkarthi commented on Sep 9, 2019

   

   tirkarthi

   on Sep 9, 2019

   Member

   More actions

   Closing this as not a bug since autospeccing with create_autospec can be used and spec only does attribute access validation.

   Thanks
9. tirkarthi

   closed this as completedon Sep 9, 2019
10. tirkarthi

    added

    invalid

    on Sep 9, 2019
11. ezio-melotti

    transferred this issue fromon Apr 10, 2022