Repository navigation
Use PyBytesWriter API in io.RawIOBase.readall #140135
Description
Feature or enhancement
Proposal:
There's new PyBytesWriter() API:
There's an ongoing effort of introducing it in various places:
With its PyList_Append
Line 1003 in 2ca3c85
and PyBytes_Join:
the io.RawIOBase.readall looks like the ideal candidate for PyBytesWriter.
Notably, io.RawIOBase is used internally across many modules:
https://github.com/search?q=repo%3Apython%2Fcpython+io.RawIOBase+path%3A%2F%5ELib%5C%2F%2F&type=code
and some even call readall explicitly sometimes:
Line 545 in 2ca3c85
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Activity
[-]Use PyBytesWriter API in `io.RawIOBase.readall`[/-][+]Use `PyBytesWriter` API in `io.RawIOBase.readall`[/+]cmaloney commentedon Oct 14, 2025
cc: @vstinner, @emmatyping . This a very similar optimization to gh-139877.
This has definitely shown up in user profiles although most the time
FileIO.readall(which is an implementation ofRawIOBase.readall) ends up getting calledI like this as an optimization although to me is an intermediate state. I really want to update this to use
bytearray+readinto+take_bytes()which does the same thing with less allocations + copies (self.read()allocates and returns a newbyteswhich is copied into thePyBytesWriterhere). I don't think that should block this optimization though. Changingio.RawIOBaseto defaulting to.readintofrom.readis unfortunately complicated to do without breaking compatibility or making infinite loops.gh-140135: Use `PyBytesWriter` in `io.RawIOBase.readall()`; 4x faster (…
pythongh-140135: Use `PyBytesWriter` in `io.RawIOBase.readall()`; 4x …