[tangentially related to CVE-2023-24329] urlparse does not correctly handle schemes that begin with ASCII digits, '+', '-', and '.' characters

Background

RFC 3986 defines a scheme like this:

scheme = ALPHA *( ALPHA / DIGIT / "+" / "-" / "." )

RFC 2234 defines an ALPHA like this:

ALPHA = %x41-5A / %x61-7A

The WHATWG URL spec defines a scheme like this:

"A URL-scheme string must be one ASCII alpha, followed by zero or more of ASCII alphanumeric, U+002B (+), U+002D (-), and U+002E (.)."

The bug

This is the scheme string parsing code from Lib/urllib/parse.py:462-468:

    i = url.find(':')
    if i > 0:
        for c in url[:i]:
            if c not in scheme_chars:
                break
        else:
            scheme, url = url[:i].lower(), url[i+1:]

This is the definition of scheme_chars from Lib/urllib/parse.py:77-80:

scheme_chars = ('abcdefghijklmnopqrstuvwxyz'
                'ABCDEFGHIJKLMNOPQRSTUVWXYZ'
                '0123456789'
                '+-.')

This will erroneously validate schemes that begin with any of ('.', '-', '+', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9'). This behavior is in violation of both specifications.

This bug is reproducible with the following snippet:

>>> from urllib.parse import urlparse
>>> urlparse(".://") # Should error, but doesn't
ParseResult(scheme='.', netloc='', path='', params='', query='', fragment='')

My environment

CPython versions tested on:
- 3.12.0a1+ (fb844e1)
- 3.10.8
Operating system and architecture:
- Arch Linux x86_64

PR: gh-99418: Make urllib.parse.urlparse enforce that a scheme must begin with an alphabetical ASCII character. #99421

PR: [3.11] gh-99418: Make urllib.parse.urlparse enforce that a scheme must begin with an alphabetical ASCII character. (GH-99421) #99446

gh-99418: Make urllib.parse.urlparse enforce that a scheme must begin…

pythongh-99418: Make urllib.parse.urlparse enforce that a scheme must…

gh-99418: Make urllib.parse.urlparse enforce that a scheme must begin…

Thanks, looks like this has been fixed

CVE-2023-24329 was assigned to this issue.

@gpshead

Python 3.7, 3.8 and 3.9 are affected by this issue and still get security fixes.

@gpshead: Should this fix be backported to Python 3.7-3.9?

Ah, I don't see a fix for Python 3.10 neither, whereas the issue was reported on Python 3.10.

I created https://python-security.readthedocs.io/vuln/urlparse-scheme.html to track fixes of this issue.

Please see #102153..

(ie: that python-security urlparse-scheme blog text is currently wrong: this is not fixed, and the first report was in July, not November)

(ie: that python-security urlparse-scheme blog text is currently wrong: this is not fixed, and the first report was in July, not November)

I'm maintaining this page manually and it's quite a lot of work to maintain it. I tried to automate as many things as possible. The source can be found in the YAML file: https://github.com/vstinner/python-security/blob/main/vulnerabilities.yaml#L2134

Free free to propose a PR to fix the entry ;-)

The fix for this CVE should really be backported if applicable.

The fix for this CVE should really be backported if applicable.

This issue does not contain the fix.

See #102153.

@gpshead

The fix for this CVE should really be backported if applicable.

This issue does not contain the fix.

See #102153.

@gpshead : thank you so very much for the pointer! I'll do some poking around next week to see if some other OS distributions have addressed this and if there aren't any available fixes, try crafting (an) appropriate patch(es) and link it/them to the appropriate issue.

I work on a project that uses 3.8; if it's too much work for 3.7, I'll just look into making the 3.8 patch work.

pythongh-99418: Make urllib.parse.urlparse enforce that a scheme must…

Navigation Menu

Repository navigation

[tangentially related to CVE-2023-24329] urlparse does not correctly handle schemes that begin with ASCII digits, '+', '-', and '.' characters #99418

Background

The bug

My environment

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

[tangentially related to CVE-2023-24329] urlparse does not correctly handle schemes that begin with ASCII digits, '+', '-', and '.' characters #99418

Description

Background

The bug

My environment

Activity

hauntsaninja commented on Nov 29, 2022

vstinner commented on Apr 5, 2023

vstinner commented on Apr 5, 2023

vstinner commented on Apr 5, 2023

vstinner commented on Apr 5, 2023

gpshead commented on Apr 8, 2023

gpshead commented on Apr 8, 2023

vstinner commented on May 4, 2023

ngie-eign commented on May 20, 2023

gpshead commented on May 20, 2023

ngie-eign commented on May 20, 2023

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions