shlex.pop_source() leaks punctuation buffered from the popped source

[augusto-rehfeldt]

Bug report

Bug description:

Documented behaviour: Python standard-library documentation, shlex.pop_source(): "Pop the last-pushed input source from the input stack." Together with shlex.push_source(): "Push an input source stream onto the input stack."

Expected: ['a']

Actual: [';', 'a']

import shlex

P, C = 'a', 'b'
if not all(s and s.isascii() and s.isalpha() for s in (P, C)):
    print('REFUTATION REJECTED: input must be nonempty ASCII letters')
else:
    expected = list(shlex.shlex(P + ' ', posix=True, punctuation_chars=';'))
    lexer = shlex.shlex(P + ' ', posix=True, punctuation_chars=';')
    lexer.push_source(C + ';tail')
    token = lexer.get_token()
    if token != C:
        print('REFUTATION REJECTED: pushed token was', repr(token))
    else:
        lexer.pop_source()
        actual = list(lexer)
        if actual != expected:
            print('REFUTATION CONFIRMED:', (P, C), 'actual:', actual, 'expected:', expected)
        else:
            print('REFUTATION REJECTED: restored source matches independent reference')

Output on Python 3.14.6 (Windows-11-10.0.26220-SP0), standard library shlex:

REFUTATION CONFIRMED: ('a', 'b') actual: [';', 'a'] expected: ['a']

This report was found and written by an automated property-testing tool I run (bugforge). The reproducer above was executed and its output is pasted unedited; no person reviewed the report before it was filed. The search script is in https://github.com/augusto-rehfeldt/bugforge-results/tree/main/shlex-20261003-015527-c4

CPython versions tested on:

3.14

Operating systems tested on:

Windows

[ZeroIntensity]

See #158748 (comment).