Port 23-argument _posixsubprocess.fork_exec to Argument Clinic

Currently the function is parsed with the following behemoth:

if (!PyArg_ParseTuple(
            args, "OOpO!OOiiiiiiiiii" _Py_PARSE_PID "OOOiOp:fork_exec",
            &process_args, &executable_list,
            &close_fds, &PyTuple_Type, &py_fds_to_keep,
            &cwd_obj, &env_list,
            &p2cread, &p2cwrite, &c2pread, &c2pwrite,
            &errread, &errwrite, &errpipe_read, &errpipe_write,
            &restore_signals, &call_setsid, &pgid_to_set,
            &gid_object, &groups_list, &uid_object, &child_umask,
            &preexec_fn, &allow_vfork))
        return NULL;

Conversion will:

hide this from a realm of manual and error-prone labor into a precise and checked world of automation
allow to use faster methods like METH_FASTCALL+_PyArg_CheckPositional.

Linked PRs

Indeed, one reason I've left it a behemoth in the past is that it only has 2-3 call-sites and gained parameters slowly over time. Cleaning this monster up will be good.

One minor concern with argument clinic on this is the C stack space use increase due to the intermediate function calling the implementation with everything as C arguments on the stack. BUT... there is some other long overdue _posixsubprocess refactoring to be done that could alleviate stack use elsewhere (not to be done as part of this issue) in 3.12 that could make up for this change by reducing a similar amount of C stack use. (the internal C call to the child fork exec function which is also a giant messy pile of C args)

Also, the argument clinic generated code is effectively a tail call to the _impl. Ideally compilers see that in optimized builds and effectively inline/merge the two functions instead of wasting time separating them with a calling convention sucking up stack. (No guarantees on that)

If we went an alternate route and constructed a structure with all of the values instead of a giant list of arguments (a namedtuple could make it an easy transition), the C stack overhead from argument clinic would be gone. But so would the nice value type checking boilerplate that argument clinic handles for us. Unless we created our own internal to _posixsubprocess type instead of namedtuple and used argument clinic on its construction and data fill-in APIs instead. That idea could just be overcomplicated. I'll ponder it.

In the meantime, thanks for the PR. I'll get to a review on that soon.

gh-94518: [_posixsubprocess] Replace variable validity flags with res…

gh-94518: Rename `group*` to `extra_group*` to avoid confusion (#101054)

pythongh-94518: Rename `group*` to `extra_group*` to avoid confusion (p…

gh-94518: Port 23-argument `_posixsubprocess.fork_exec` to Argument C…

thanks! :)

Merge branch 'main' into superopt

Merge branch 'superopt' into superopt_spec

Navigation Menu

Repository navigation

Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic #94518

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Port 23-argument _posixsubprocess.fork_exec to Argument Clinic #94518

Description

Linked PRs

Activity

gpshead commented on Jul 3, 2022

gpshead commented on Apr 24, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Port 23-argument `_posixsubprocess.fork_exec` to Argument Clinic #94518