Iterating over ChainMap.values() and ChainMap.items() updates underlying defaultdicts

Bug report

Hello,
I encountered some confusing behavior when combining ChainMap with defaultdicts. Might be related to #23534
Specifically, iterating over the values or items of the ChainMap updates one of the underlying dictionaries, and returns the wrong result.
Just iterating over they keys or the dictionary directly produces the right results still and doesn't update the underlying dictionaries.

from collections import ChainMap, defaultdict

d1 = defaultdict(int, {'a': 1, 'b': 2})
d2 = defaultdict(int, {'c': 3, 'd': 4})
d3 = ChainMap(d1, d2)

print('Prior to iter')
print(d1)
print(d2)
print('After iter')
a = list(d3.values()) # Same result when calling d3.items
print(a) # Should return: [1, 2, 3, 4]. Returns: [0, 0, 1, 2]
print(d1) # Now: {'a': 1, 'b': 2, 'c': 0, 'd': 0}
print(d2) # Unchanged

Your environment

Tested on Python version 3.10.4.
Linux 5.13.0-41-generic #46~20.04.1-Ubuntu SMP
x86_64 x86_64 x86_64 GNU/Linux

@serhiy-storchaka

cc @serhiy-storchaka

@rhettinger

~~The following patch will solve this issue, but I am waiting for @rhettinger and @serhiy-storchaka~~

(Sorry for editing the previous comment: it will break other behavior.)

Lookups search the underlying mappings successively until a key is found. ¹

This behavior looks like it is intended because this is a matter of look-up order.
IMO, under the script, d1 did what it should do as defaultdict when meeting the non-existence key.
but I am waiting for both of the members whom I mentioned.

https://docs.python.org/3/library/collections.html?#collections.ChainMap ↩

Yes, it looks like an intentional behavior. There is even a comment in the code about this:

cpython/Lib/collections/__init__.py

Line 1001 in 8150b8c

return mapping[key] # can't use 'key in mapping' with defaultdict

To the extent that this is a problem, it lies with the defaultdict updating as a side-effect of lookups. If that isn't desired, it isn't hard to define a dict subclass with __missing__ that doesn't have the updating behavior (see collections.Counter for an example of how to do this).

FWIW, you can turn-off defaultdict's updating behavior prior to iteration:

d1.default_factory = None
d2.default_factory = None
a = list(d3.values()) # This no longer mutates d1 and d2

Navigation Menu

Repository navigation

Iterating over ChainMap.values() and ChainMap.items() updates underlying defaultdicts #93432

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Iterating over ChainMap.values() and ChainMap.items() updates underlying defaultdicts #93432

Description

Activity

corona10 commented on Jun 4, 2022

corona10 commented on Jun 4, 2022

Footnotes

serhiy-storchaka commented on Jun 4, 2022

rhettinger commented on Jun 5, 2022

rhettinger commented on Jul 6, 2022

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions