Skip to content

Use PyBytesWriter API in PEG parser's _build_concatenated_bytes, avoid quadratic memory allocations #140149

Closed

Description

@maurycy

Feature or enhancement

Proposal:

There's new PyBytesWriter() API:

There's an ongoing effort of introducing it in various places:

action_helpers.c's _build_concatenated_bytes, which is a part of the PEG parser, is a great candidate for using PyBytesWriter API with its PyBytes_Concat:

PyBytes_Concat(&res, elem->v.Constant.value);

This seems to be quadratic time complexity, while using a modern API and just a single copy is possible!

Inspired by gh-140139, I found this while grepping for PyBytes_Concat and PyUnicodeWriter. My hypothesis is that for many there's a missing PyBytesWriter yang.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

Activity

  1. added
    interpreter-core(Objects, Python, Grammar, and Parser dirs)
    type-refactorCode refactoring (with no changes in behavior)
    and removed
    type-featureA feature request or enhancement
    on Oct 15, 2025
  2. added a commit that references this issue on Oct 16, 2025
  3. added a commit that references this issue on Dec 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-parsertype-refactorCode refactoring (with no changes in behavior)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions