Repository navigation
zipfile.Path has only partial compatibility io.BytesIO #130120
Copy link
Copy link
Closed as duplicate
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Description
Bug description:
I encountered an edge-case while using zipfile.Path with in-memory data buffers. It seems like the filename of the root is set to None, which breaks anything that tries to join the zip-filename and its internal path (including str(), ``.
I have two suggested fixes:
- Update the initialization of the
zipfile.FastLookup-class and setself.filenameto":memory:"if it is None, matching what you would write in e.g. sqlite to get an in-memory database. - Update all places where the code attempts do Path-operations on
self.root.filenameto notice if the filename is None and use ":memory:" instead.
The first is definitely the easiest fix, but the second would maybe be more backwards compatible in case someone uses Path.root.filename to notice that the zipfile.Path-object points to an in-memory zipfile. Though, I'm not sure how big of a problem that is.
I don't have any strong opinions of which to choose, and I would be happy to implement the fix.
Minimal reproducible example
import io
import zipfile
# Create a dummy Zip file
data = io.BytesIO()
with zipfile.ZipFile(data, mode="w") as zf:
zf.writestr("hello.txt", "python\n")
# Try to iterate over its content
data.seek(0)
p = zipfile.Path(data)
print(str(p))
print(list(p.iterdir()))---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
Cell In[92], line 13
11 p = zipfile.Path(data)
---> 12 print(str(p))
File /usr/lib/python3.12/zipfile/_path/__init__.py:391, in Path.__str__(self)
390 def __str__(self):
--> 391 return posixpath.join(self.root.filename, self.at)
File <frozen posixpath>:76, in join(a, *p)
TypeError: expected str, bytes or os.PathLike object, not NoneType
Demonstration of fix
import io
import zipfile
# Create a dummy Zip file
data = io.BytesIO()
with zipfile.ZipFile(data, mode="w") as zf:
zf.writestr("hello.txt", "python\n")
# Try to iterate over its content
data.seek(0)
p = zipfile.Path(data)
p.root.filename = ":memory:"
print(str(p))
print(list(p.iterdir())):memory:/
[Path(':memory:', 'hello.txt')]
CPython versions tested on:
3.12, 3.14
Operating systems tested on:
Ubuntu 24.04 (WSL), Fedora (via the Python devcontainer on GitHub codespaces)
Linked PRs
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
Projects
- StatusShow more project fieldsDone
Activity
yngvem commentedon Feb 14, 2025
I just realised that option 2. might be preferable, because we should probably overload the
parentmethod to prevent it from resolving to"."when the path is":memory:"anyways?yngvem commentedon Feb 20, 2025
I've started working on this now. I had one idea though: How about setting the name to
\0if it represents an in-memory byte string? That way it's clear that it isn't a local path with the name":memory:".Update: Looking at the tests, it seems like it's supposed to fail. So I'll only update the code so
__str__doesn't fail