Argument Clinic should understand *args and **kwargs parameters

@rhettinger

BPO	20291
Nosy	@rhettinger, @ncoghlan, @vstinner, @larryhastings, @methane, @meadori, @zware, @serhiy-storchaka, @GPHemsley, @isidentical, @Fidget-Spinner, @colorfulappl
PRs	gh-64490: varargs support for argument clinic and refactor print with AC #18609 bpo-20291: Fix MSVC warnings in getargs.c #27211 gh-90350: Optimize builtin functions min() and max() #30286 gh-64490: Fix bugs in argument clinic varargs processing #32092

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = None
created_at = <Date 2014-01-18.01:55:25.863>
labels = ['type-feature', '3.7', 'expert-argument-clinic']
title = 'Argument Clinic should understand *args and **kwargs parameters'
updated_at = <Date 2022-03-24.08:27:39.531>
user = 'https://github.com/larryhastings'

bugs.python.org fields:

activity = <Date 2022-03-24.08:27:39.531>
actor = 'colorfulappl'
assignee = 'serhiy.storchaka'
closed = False
closed_date = None
closer = None
components = ['Demos and Tools', 'Argument Clinic']
creation = <Date 2014-01-18.01:55:25.863>
creator = 'larry'
dependencies = []
files = []
hgrepos = []
issue_num = 20291
keywords = ['patch']
message_count = 10.0
messages = ['208380', '225514', '225531', '264959', '285642', '285647', '285654', '285660', '285779', '397721']
nosy_count = 12.0
nosy_names = ['rhettinger', 'ncoghlan', 'vstinner', 'larry', 'methane', 'meador.inge', 'zach.ware', 'serhiy.storchaka', 'gphemsley', 'BTaskaya', 'kj', 'colorfulappl']
pr_nums = ['18609', '27211', '30286', '32092']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue20291'
versions = ['Python 3.7']

PR: gh-64490: Fix refcount error when arguments are packed to tuple in argument clinic #99233

PR: [3.11] gh-64490: Fix refcount error when arguments are packed to tuple in argument clinic (GH-99233) #100338

PR: [3.11] gh-64490: Fix bugs in argument clinic varargs processing (GH-32092) #100368

PR: gh-64490: Argument Clinic: Add support for **kwds #138344

PR: gh-64490: Tweak AC implementation for **kwds #139133

Argument Clinic currently prevents the "impl" function from ever seeing the "args" tuple or the "kwargs" dict. There should be a way to ask it to pass those values in to the "impl" function.

So, let's think about this for a minute. What's the API that we *want* here?

If your function has the signature
(a, b, c=20, *args)
and you call it with
(1, 2, 3, 4, 5)
should "args" be (4, 5), or (1, 2, 3, 4, 5)?

I assert that the impl function should get the same "args" (and "kwargs") that a Python function would--that is, post-argument-processing. In the above example "args" should get (4, 5).

This might be somewhat painful to do in round 1, where we're still leveraging off PyArg_ParseTuple*. But in the future it'll be cheaper to do it this way. In any case, it's the right API, so that's what we should do.

(Adding Nick just to see if he agrees--he had a use case for *args in the builtin module.)

Yes, I agree we should follow the Python level semantics, and only capture the excess positional arguments. For the record, the four builtins I flagged as needing this in order to add introspection information:

__build_class__

2 positional only args, arbitrary additional args
checks size with PyTuple_GET_SIZE
uses PyTuple_GET_ITEM x2 + PyTuple_GetSlice to unpack

print

only arbitrary position args, iterates and extracts using PyTuple_GetItem
uses PyArg_ParseTupleAndKeywords with an empty tuple to extract the keyword-only args

min
max

use a shared helper function min_max
uses the args tuple directly if size > 1
otherwise uses PyArg_UnpackTuple to extract the supplied iterable
uses PyArg_ParseTupleAndKeywords with an empty tuple to extract the keyword-only args
this "one arg means iterable" style API might need to be a special case...

I think at first step we can support var-positional parameter only when there are no other positional parameters, and var-keyword parameter only when there are no other keyword parameters. So print, max and dict.update will be supported, but __build_class__, map and functools.partial are not.

I'll try to implement the support of var-positional parameters.

In case it is helpful, here's my list of examples where the AC and existing signature objects are insufficiently expressive:

type(obj)
type(name, bases, mapping)
    two different signatures depending on type   

range(stop)
range(start, stop)
range(start, stop, step)

dict.pop(key[, default])
   default of None has different meaning than missing default
   which raises KeyError when the key is missing

itertools.permutations(iterable[, r])
   where the absence of *r* implies r=len(iterable)

bisect.bisect_right(a, x[, lo[, hi]]) -> index
where the absence of *hi* implies hi=len(a)

min(iterable, *[, default=obj, key=func]) -> value
min(arg1, arg2, *args, *[, key=func]) -> value
has two signatures depending on the number of
positional arguments and a keyword argument
only used in the first signature. It's implementation
is also shared with max().

dict() -> new empty dictionary
dict(mapping) -> new dictionary initialized from a mapping object's
(key, value) pairs
dict(iterable) -> new dictionary initialized as if via:
d = {}
for k, v in iterable:
d[k] = v
dict(**kwargs) -> new dictionary initialized with the name=value pairs
in the keyword argument list. For example: dict(one=1, two=2)

def sumseq(seq, a=0, b=None):
    # Pure python code with nullable int
    if b is None:
        b = len(seq)
    return sum(seq[a:b])

FYI I started to work on a different Argument Clinic enhancement, issue bpo-29299: "Argument Clinic: Fix signature of optional positional-only arguments".

gh-64490: Fix refcount error when arguments are packed to tuple in ar…

gh-64490: Fix bugs in argument clinic varargs processing (#32092)

[3.11] pythongh-64490: Fix refcount error when arguments are packed t…

[3.11] gh-64490: Fix refcount error when arguments are packed to tupl…

[3.11] pythongh-64490: Fix bugs in argument clinic varargs processing (…

[3.11] gh-64490: Fix bugs in argument clinic varargs processing (GH-3…

#18609 appears to have added *args support, but it wasn't documented.

Doc/howto/clinic.rst could use an update.

@isidentical

(unassigning since it isn't actively being worked on) @isidentical as FYI for the missing documentation.

#18609 appears to have added *args support, but it wasn't documented.

See python/devguide#1629

I've had a stab at **kwds support, see #138344. It is an intentionally restricted implementation, only supporting METH_VARARGS|METH_KEYWORDS and strictly positional-ponly parameters. However, I think it's a useful start, and allows converting several functions to AC already, which affords us the other benefits of using AC for those functions.

A

gh-64490: Argument Clinic: Add support for ``**kwds`` (#138344)

There was a bug:

Argument Clinic reads a missing optional positional argument in functions with **kwds #156219

gh-64490: Tweak AC implementation for **kwds (GH-139133)

Navigation Menu

Repository navigation

Argument Clinic should understand *args and **kwargs parameters #64490

4 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Argument Clinic should understand *args and **kwargs parameters #64490

Description

Activity

larryhastings commented on Jan 18, 2014

larryhastings commented on Aug 19, 2014

ncoghlan commented on Aug 19, 2014

serhiy-storchaka commented on May 6, 2016

serhiy-storchaka commented on Jan 17, 2017

rhettinger commented on Jan 17, 2017

vstinner commented on Jan 17, 2017

4 remaining items

gpshead commented on Feb 7, 2023

gpshead commented on Feb 7, 2023

skirpichev commented on Aug 12, 2025

AA-Turner commented on Sep 1, 2025

serhiy-storchaka commented on Aug 22, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions