Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute

@tiran

BPO	35665
Nosy	@tiran, @pervlad, @MDM-79, @pukkandan
Files	cacerts.pem: pub certs Untitled.png: MUPCA Root - Certificates

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-01-08.09:38:24.479>
created_at = <Date 2019-01-05.12:24:45.346>
labels = ['expert-SSL', 'type-bug', '3.9']
title = 'Function ssl.create_default_context raises exception on Windows 10  when called with  ssl.Purpose.SERVER_AUTH) attribute'
updated_at = <Date 2021-09-28.21:47:58.903>
user = 'https://github.com/pervlad'

bugs.python.org fields:

activity = <Date 2021-09-28.21:47:58.903>
actor = 'pukkandan'
assignee = 'none'
closed = True
closed_date = <Date 2019-01-08.09:38:24.479>
closer = 'pervlad'
components = ['SSL']
creation = <Date 2019-01-05.12:24:45.346>
creator = 'pervlad'
dependencies = []
files = ['48029', '50311']
hgrepos = []
issue_num = 35665
keywords = []
message_count = 16.0
messages = ['333054', '333055', '333158', '333159', '333168', '333181', '333183', '333184', '333215', '333224', '360050', '402782', '402792', '402794', '402805', '402809']
nosy_count = 5.0
nosy_names = ['christian.heimes', 'pervlad', 'pedja', 'MDM-1', 'pukkandan']
pr_nums = []
priority = 'normal'
resolution = 'third party'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35665'
versions = ['Python 3.9']

Linked PRs

In Python 3.7.1 on Windows 10 ssl library function call ssl.create_default_context(ssl.Purpose.SERVER_AUTH) raises an ssl error:

File "C:\Python37\lib\ssl.py", line 471, in _load_windows_store_certs
self.load_verify_locations(cadata=certs)
ssl.SSLError: nested asn1 error (_ssl.c:3926)

In Python 3.6.4 same function call raises no error.

Same outcome in Python 3.7.2.
See first comment for detailed explanation of issue.

self.load_verify_locations(cadata=certs)
...
ssl.SSLError: nested asn1 error (_ssl.c:3926)

It seems like one of your certificate is invalid.

In Python 3.6.4 same function call raises no error.

We frequently update OpenSSL in Python. You can get OpenSSL version using:

$ python3
Python 3.7.2 (default, Jan  3 2019, 09:14:01) 
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.1 FIPS  11 Sep 2018'
>>> ssl.OPENSSL_VERSION_INFO
(1, 1, 1, 0, 15)
>>> ssl.OPENSSL_VERSION_NUMBER
269488143
>>> hex(ssl.OPENSSL_VERSION_NUMBER)
'0x1010100f'

Would it be possible to attach the certification to the issue so someone can try to reproduce the issue? (but don't attach any private key ;-))

The certs are coming from Windows' trust store. Could you please dump the trust store for me and attach the result to the bug tracker. The following script is untested but should work. I don't have access to a Windows machine at the moment.

ctx = ssl.SSLContext(ssl.PROTOCOL_TLS)
certs = []
for storename in ("CA", "ROOT"):
    certs.append(storename)
    for cert, encoding, trust in ssl.enum_certificates(storename):
        if encoding == "x509_asn":
            if trust is True or ssl.Purpose.SERVER_AUTH.oid in trust:
                try:
                    ctx.load_verify_locations(cadata=cert)
                except Exception as e:
                    certs.append(str(e))
                certs.append(ssl.DER_cert_to_PEM_cert(cert))

with open('cacerts.pem', 'w') as f:
    f.write('\n'.join(certs))

Public Certificate file cert.pem is attached.

Version of ssl lib in pythons on my machine:
Python 3.7.2 (tags/v3.7.2:9a3ffc0492, Dec 23 2018, 23:09:28) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.1.0j  20 Nov 2018'

Python 3.6.8 (tags/v3.6.8:3c6b436a57, Dec 24 2018, 00:16:47) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ssl.OPENSSL_VERSION
'OpenSSL 1.0.2q  20 Nov 2018'

Your Windows cert store contains multiple invalid certificates. The first failing certificate is the custom "MUPCA Root", which looks like a certificate from http://ca.mup.gov.rs/sertifikati.html. The serial number seems to be badly formated or padded. There is nothing we can do about erroneous and bad certificates.

$ openssl x509 -in ca.pem 
unable to load certificate
140613019477824:error:0D0E20DD:asn1 encoding routines:c2i_ibuf:illegal padding:crypto/asn1/a_int.c:187:
140613019477824:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=serialNumber, Type=X509_CINF
140613019477824:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=cert_info, Type=X509
140613019477824:error:0906700D:PEM routines:PEM_ASN1_read_bio:ASN1 lib:crypto/pem/pem_oth.c:33:

$ openssl asn1parse -in ca.pem  
    0:d=0  hl=4 l=1300 cons: SEQUENCE          
    4:d=1  hl=4 l= 764 cons: SEQUENCE          
    8:d=2  hl=2 l=   3 cons: cont [ 0 ]        
   10:d=3  hl=2 l=   1 prim: INTEGER           :02
   13:d=2  hl=2 l=   4 prim: INTEGER           :BAD INTEGER:[00000066]
   19:d=2  hl=2 l=  13 cons: SEQUENCE          
   21:d=3  hl=2 l=   9 prim: OBJECT            :sha1WithRSAEncryption
   32:d=3  hl=2 l=   0 prim: NULL              
   34:d=2  hl=2 l=  83 cons: SEQUENCE          
   36:d=3  hl=2 l=  19 cons: SET               
   38:d=4  hl=2 l=  17 cons: SEQUENCE          
   40:d=5  hl=2 l=   3 prim: OBJECT            :commonName
   45:d=5  hl=2 l=  10 prim: UTF8STRING        :MUPCA Root
   57:d=3  hl=2 l=  29 cons: SET               
   59:d=4  hl=2 l=  27 cons: SEQUENCE          
   61:d=5  hl=2 l=   3 prim: OBJECT            :organizationName
   66:d=5  hl=2 l=  20 prim: UTF8STRING        :MUP Republike Srbije
   88:d=3  hl=2 l=  16 cons: SET               
   90:d=4  hl=2 l=  14 cons: SEQUENCE          
   92:d=5  hl=2 l=   3 prim: OBJECT            :localityName
   97:d=5  hl=2 l=   7 prim: UTF8STRING        :Beograd
  106:d=3  hl=2 l=  11 cons: SET               
  108:d=4  hl=2 l=   9 cons: SEQUENCE          
  110:d=5  hl=2 l=   3 prim: OBJECT            :countryName
  115:d=5  hl=2 l=   2 prim: PRINTABLESTRING   :RS
  119:d=2  hl=2 l=  30 cons: SEQUENCE          
  121:d=3  hl=2 l=  13 prim: UTCTIME           :100227161918Z
  136:d=3  hl=2 l=  13 prim: UTCTIME           :200227161918Z
  ...

$ wget http://ca.mup.gov.rs/MUPCARoot.crt
$ openssl x509 -in MUPCARoot.crt -inform DER
unable to load certificate
140699773712192:error:0D0E20DD:asn1 encoding routines:c2i_ibuf:illegal padding:crypto/asn1/a_int.c:187:
140699773712192:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=serialNumber, Type=X509_CINF
140699773712192:error:0D08303A:asn1 encoding routines:asn1_template_noexp_d2i:nested asn1 error:crypto/asn1/tasn_dec.c:627:Field=cert_info, Type=X509

OpenSSL 1.1.0 is more strict than OpenSSL 1.0.2. That's why you don't see the issue with Python 3.6 but with 3.7. The problem is explained in https://mta.openssl.org/pipermail/openssl-dev/2016-February/005100.html

The CA has encoded the integer 102 (0x66) as "02 04 00
00 00 66", which violates the DER standard. The correct encoding is "02 01 66".

>>> from asn1crypto.core import Integer
>>> import binascii
>>> binascii.hexlify(Integer(102).dump())
b'020166'

Thank you all for this expeditive help. Sorry for taking your time.
I will remove bad certificates from my machine.
Thanks again.

I will try to close this one.

I also checked how other implementations deal with invalid DER encoding. NSS 3.41, Firefox, and Chromium accept the certifiate.

NSS shows the serial number as "102 (0x66)"
Firefox and Chromium display the serial number as "00:00:00:66".

$ echo "password" > passwd
$ certutil -d . -f passwd -N
$ certutil -d . -f passwd -A -n ca -i ../ca.pem -t C,C,C
$ certutil -d . -L -n ca
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 102 (0x66)
        Signature Algorithm: PKCS #1 SHA-1 With RSA Encryption
        Issuer: "C=RS,L=Beograd,O=MUP Republike Srbije,CN=MUPCA Root"
        Validity:
            Not Before: Sat Feb 27 16:19:18 2010
            Not After : Thu Feb 27 16:19:18 2020
        Subject: "C=Re...,L=Beograd,O=MUP Republike Srbije,CN=MUPCA Resursi"
        Subject Public Key Info:
            Public Key Algorithm: PKCS #1 RSA Encryption
            RSA Public Key:
                Modulus:
                    ea:69:46:bc:c7:70:00:d5:f5:32:8d:c7:4e:ad:3a:a5:
                    d3:29:7e:a2:46:12:a9:dd:57:75:b1:49:95:80:20:ed:
                    9b:68:6b:e3:c5:55:d8:64:15:68:42:ab:a3:f7:c0:96:
                    37:08:51:cb:05:ca:b5:99:f6:07:a6:8b:f2:cd:d2:f5:
                    d6:16:12:da:bf:a8:0b:9c:45:5d:ac:79:1d:a8:67:47:
                    ee:7f:83:40:f8:58:00:d5:dd:c4:c9:52:1b:d2:f4:ce:
                    e1:fa:8a:66:d3:18:86:1e:ea:fc:0a:8b:b5:ec:49:cd:
                    86:bf:8b:7e:b0:61:81:ec:ea:99:4f:64:82:96:93:9d:
                    ab:80:7d:a7:27:65:00:d4:12:26:98:45:64:7e:76:0b:
                    98:ff:16:50:49:0c:45:20:82:ce:2e:23:a2:65:3a:b7:
                    44:cd:51:00:d9:bf:e3:1f:de:23:1d:57:e9:32:c3:55:
                    f0:24:af:d4:cf:cd:9e:77:1f:19:7e:1c:03:5b:7a:e4:
                    75:84:3b:d4:1d:e9:23:d6:8c:f2:8f:b2:0d:e3:79:df:
                    9e:03:1e:0e:15:5b:7b:0c:dd:6e:4d:82:86:5a:63:79:
                    64:b5:07:79:dd:fd:08:e3:d6:cb:60:01:fd:82:11:59:
                    2c:8d:22:f8:f9:91:59:b1:cd:12:7b:39:6d:08:82:5d
                Exponent: 65537 (0x10001)
        Signed Extensions:
            Name: Certificate Basic Constraints
            Critical: True
            Data: Is a CA with no maximum path length.

        Name: Certificate Key Usage
        Critical: True
        Usages: Certificate Signing
                CRL Signing

        Name: Authority Information Access
        Method: PKIX CA issuers access method
        Location: 
            URI: "http://ca.mup.gov.rs/MUPCARoot.crt"

        Name: Certificate Subject Key ID
        Data:
            cb:f9:00:a9:b7:b6:c1:6f:44:43:d0:22:ad:fc:0e:6e:
            cc:8f:f6:0f

        Name: Certificate Authority Key Identifier
        Key ID:
            3f:66:b0:0f:66:fb:f0:10:2e:61:a4:6f:ef:2c:95:8a:
            14:72:6f:71

        Name: CRL Distribution Points
        Distribution point:
            URI: "http://ca.mup.gov.rs/MUPCARoot.crl"

Signature Algorithm: PKCS #1 SHA-1 With RSA Encryption

gh-79846: Make ssl.create_default_context() ignore invalid certificat…

pythongh-79846: Make ssl.create_default_context() ignore invalid cert…

@levicki

Thank you @levicki for reviving this issue.

@levicki

Thank you @levicki for reviving this issue.

You're welcome.

However, if the fix isn't backported to 3.10 and 3.11 as well you may keep receiving those bug reports — many popular "AI" repos (such as AUTOMATIC1111 for example) have 3.10 as a requirement. Some of them can work also with 3.11, but many can't upgrade without breaking metric ton of libraries they use.

@levicki

@levicki Unfortunately 3.10 and 3.11 are only accepting security fixes, see https://devguide.python.org/versions/ and https://devguide.python.org/developer-workflow/development-cycle/#security-branches.

@serhiy-storchaka The 3.12 and 3.13 backports are pending merge, keeping in mind 3.13 is in the RC phase.

[3.12] gh-79846: Make ssl.create_default_context() ignore invalid cer…

@hugovk

@hugovk

@levicki Unfortunately 3.10 and 3.11 are only accepting security fixes, see https://devguide.python.org/versions/ and https://devguide.python.org/developer-workflow/development-cycle/#security-branches.

Yes, I am well aware of that. Too bad, because 3.10 is still widely used in AI community without any indication that it could be replaced soon due to all code dependencies.

pythongh-79846: Make ssl.create_default_context() ignore invalid cert…

[3.13] gh-79846: Make ssl.create_default_context() ignore invalid cer…

Navigation Menu

Repository navigation

Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute #79846

Linked PRs

18 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute #79846

Description

Linked PRs

Activity

pervlad commented on Jan 5, 2019

pervlad commented on Jan 5, 2019

vstinner commented on Jan 7, 2019

vstinner commented on Jan 7, 2019

tiran commented on Jan 7, 2019

pervlad commented on Jan 7, 2019

tiran commented on Jan 7, 2019

tiran commented on Jan 7, 2019

pervlad commented on Jan 8, 2019

tiran commented on Jan 8, 2019

18 remaining items

serhiy-storchaka commented on Aug 7, 2024

levicki commented on Aug 7, 2024

hugovk commented on Aug 9, 2024

levicki commented on Aug 9, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions