Skip to content

sys.flags.gil should be a "sequence" attribute OR stop adding to the sequence #122575

Closed
Closed
sys.flags.gil should be a "sequence" attribute OR stop adding to the sequence#122575
Labels
3.13only security fixes3.14bugs and security fixestype-bugAn unexpected behavior, bug, or error

Description

@colesbury
colesbury
opened on Aug 1, 2024

Bug report

sys.flags is a PyStructSequence. PyStructSequence is similar to a named tuple, but it can have attributes that are not part of the sequence.

Currently, sys.flags.gil is not a "sequence" attribute:

>>> import sys
>>> sys.flags
sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0, dev_mode=False, utf8_mode=0, warn_default_encoding=0, safe_path=False, int_max_str_digits=4300)

I think this was an oversight. We forgot to update the n_in_sequence field from 18 to 19 in gh-116338:

cpython/Python/sysmodule.c

Lines 3123 to 3128 in fda6bd8

static PyStructSequence_Desc flags_desc = {
"sys.flags", /* name */
flags__doc__, /* doc */
flags_fields, /* fields */
18
};

Linked PRs

Activity

  1. colesbury
    added
    type-bugAn unexpected behavior, bug, or error
    3.13only security fixes
    3.14bugs and security fixes
    on Aug 1, 2024
  2. colesbury
    added a commit that references this issue on Aug 1, 2024

    pythongh-122575: Include `sys.flags.gil` as a sequence attribute

    cfaa04f

  3. encukou commented on Aug 2, 2024

    @encukou
    encukou
    on Aug 2, 2024
    Member

    Generally, changing n_in_sequence is a backwards-incompatible change: code that unpacks the tuple will break.
    With 18 elements, and after int_max_str_digits was added in a bugfix release, I guess no one unpacks sys.flags specifically.
    Still, it might be better to change PyStructSequence.__repr__ than to allow sys.flags[18]?

  4. colesbury commented on Aug 2, 2024

    @colesbury
    colesbury
    on Aug 2, 2024
    ContributorAuthor

    Generally, changing n_in_sequence is a backwards-incompatible change...

    That seems like an entirely hypothetical concern. As you wrote, I don't think anyone unpacks the 18 or 19 elements of sys.flags. The longstanding practice is to include added elements. The last time this we did this was when we added int_max_str_digits in 3.12, which not only was included as a positional element, but backported all the way back to 3.7:

    This is the only PyStructSequence that has a non-sequence element, and the omission was an oversight, not intentional.

    Still, it might be better to change PyStructSequence.__repr__

    I don't think so. These are publicly documented as "named tuples" and named tuples do not support this feature of attributes that are only accessible by name.

  5. encukou commented on Aug 2, 2024

    @encukou
    encukou
    on Aug 2, 2024
    Member

    Here's what I think. Perhaps this needs wider discussion.

    This is the only PyStructSequence that has a non-sequence element

    That's not the case:

    >>> import os
>>> os.stat('.').n_fields
19
>>> os.stat('.').n_sequence_fields
10

    The last time this we did this was when we added int_max_str_digits in 3.12, which not only was included as a positional element, but backported all the way back to 3.7

    Sadly, yes. The security fix didn't go through public review.
    As far as I can tell, the next-to-last time we did this was in 2019. In my view, both of those were oversights.

    I don't think so. These are publicly documented as "named tuples" and named tuples do not support this feature of attributes that are only accessible by name.

    Note that the whole concept name-only attributes was added to PyStructSequence specifically in order to not break unpacking, back when PyStructSequence was added to replace real tuples: #35189 (comment)
    (If it wasn't for backwards compatibility with older tuple API, structseqs would probably just be “structs”; not iterable at all.)

    Sadly, the user-facing documentation for creating these types doesn't mention how they should be used in stdlib.

  6. colesbury commented on Aug 2, 2024

    @colesbury
    colesbury
    on Aug 2, 2024
    ContributorAuthor

    That's not the case ...

    Thanks, I misinterpreted what I saw in gh-122577 and missed that.

    As far as I can tell, the next-to-last time we did this was #13488

    We've added flags over a dozen times before that. Every time we've updated n_in_sequence or fixed it up in a subsequent commit (e.g., 90e8f8c)

    If you want to implement this differently, that's fine with me, but I think it's a waste of time.

  7. vstinner commented on Sep 2, 2024

    @vstinner
    vstinner
    on Sep 2, 2024
    Member

    Generally, changing n_in_sequence is a backwards-incompatible change: code that unpacks the tuple will break.

    Over the last years, many flags were added to sys.flags and I don't recall a single bug report.

    I'm convinced that nobody is unpacking sys.flags. It sounds boring to write flag1, flag2, flag3, ..., flag18 = sys.flags, while you can just write value = sys.flags[12].

  8. encukou commented on Oct 16, 2024

    @encukou
    encukou
    on Oct 16, 2024
    Member

    Unfortunately I haven't had time to dig deeper. Don't take my concern too seriously.

  9. vstinner commented on Oct 16, 2024

    @vstinner
    vstinner
    on Oct 16, 2024
    Member

    Python 3.13.0 is now shipped without sys.flags[18]. IMO it's now too late to change it.

    $ ./python
Python 3.13.0+ (heads/3.13:4eab6e8d296, Oct  8 2024, 13:44:17) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)] on linux
>>> import sys
>>> sys.flags
sys.flags(..., int_max_str_digits=4300)
>>> len(sys.flags)
18
>>> sys.flags[17]  # int_max_str_digits
4300

>>> sys.flags[18]
IndexError: tuple index out of range
>>> sys.flags.gil
1

  10. vstinner commented on Oct 16, 2024

    @vstinner
    vstinner
    on Oct 16, 2024
    Member

    IMO we should deprecate the tuple API for sys.flags. It sounds really bad to rely on an exact flag "index" to get a flag, instead of using its name.

    Good example:

    >>> sys.flags.safe_path
False

    Bad example:

    >>> sys.flags[16]
False
  11. gpshead
    added a commit that references this issue on Mar 15, 2026

    pythongh-122575: pythongh-142349: fix sys.flags tuple size (it uninte…

    c152fbd

  12. gpshead commented on Mar 15, 2026

    @gpshead
    gpshead
    on Mar 15, 2026
    Member

    I'm locking down the sequence length for sys.flags in #145988 - new attributes are now name only. the lazy imports addition PR accidentally incremented the sequence length (exposing sys.flags.gil as [18], not exposing any of the other 3 fields added after that), the PR undoes that and updates tests and code comments to make it clear that we don't want new indices.

  13. gpshead
    added a commit that references this issue on Mar 15, 2026

    gh-122575: gh-142349: fix sys.flags tuple size (it unintentionally in…

    4a71946
  14. gpshead
    changed the title [-]`sys.flags.gil` should be a "sequence" attribute[/-] [+]`sys.flags.gil` should be a "sequence" attribute OR stop adding to the sequence[/+] on Mar 15, 2026
  15. gpshead
    closed this as completedon Mar 15, 2026
  16. ljfp
    added a commit that references this issue on Apr 25, 2026

    pythongh-122575: pythongh-142349: fix sys.flags tuple size (it uninte…

    4c6f1c5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.13only security fixes3.14bugs and security fixestype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions