configparser accepts invalid keys and sections when writing

@ambv

BPO	21498
Nosy	@ambv, @The-Compiler, @isidentical

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/ambv'
closed_at = None
created_at = <Date 2014-05-13.14:47:29.111>
labels = ['type-bug', 'library']
title = 'configparser accepts keys beginning with comment_chars when writing'
updated_at = <Date 2020-01-11.15:02:37.046>
user = 'https://github.com/The-Compiler'

bugs.python.org fields:

activity = <Date 2020-01-11.15:02:37.046>
actor = 'BTaskaya'
assignee = 'lukasz.langa'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2014-05-13.14:47:29.111>
creator = 'The Compiler'
dependencies = []
files = []
hgrepos = []
issue_num = 21498
keywords = []
message_count = 2.0
messages = ['218463', '359797']
nosy_count = 3.0
nosy_names = ['lukasz.langa', 'The Compiler', 'BTaskaya']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue21498'
versions = []

Linked PRs

When adding something to a configparser instance which has a key beginning with a comment char, it writes the data to a file without generating an error, and when reading the file back obviously the data is different as it's a comment:

>>> import configparser, sys
>>> cp = configparser.ConfigParser()
>>> cp.read_dict({'DEFAULT': {';foo': 'bar'}})
>>> cp.write(sys.stdout)
[DEFAULT]
;foo = bar

This was discussed on python-dev here:
https://mail.python.org/pipermail/python-dev/2014-April/134293.html

Of course there are other corner cases as well, like having a key like "[foo]" or "=bar".

I think whatever data I pass into a configparser should also come out again when reading the file back.

Since there's no escaping in configparser, I think the ideal solution would be configparser refusing to write ambigious values.

While this is technically a backwards-incompatible change, applications doing this were broken in the first place, so validation while writing will not break anything.

Validating when setting values would be better of course, but this can potentially break applications where configparser is used without actually writing a file.

Any update on this? Discussion over Python-Dev looks like finished without a consensus/resolution.

#69909 is similar - and much more detailed.

See also #69909, #67490, #65122, where the prior consensus seems to have been to be lenient to what the user provides, and leave it up to them to provide valid inputs. I can see the flip side that it would be valuable to provide some level of validation and reasonableness checks. However, it appears there are cases where other parsers might accept inputs that Python's configparser might not accept (e.g. section names like [Test[2]_foo] as mentioned in #69909), where it might be valuable for a Python script to be able to generate such outputs even if it can't read it. It won't be possible with a simple implementation to please both users that want to generate idiosyncratic output and those that want to reject it.

I've re-classified this as a feature and not a bug as the current approach is stable and defended and any change is going to be some kind of enhancement.

I think the next thing that needs to happen is to draft a design that addresses the concerns laid out in the various discussions. Such a design should probably start by synthesizing all of the concerns brought up in these different threads (the issues mentioned above and other discussions linked from those). An LLM might be helpful in performing such a synthesis.

I'm happy to sponsor this work (guide, direct, mentor), but I won't be working on it myself and I don't have a strong opinion on the design. The key is going to be to garner consensus.

where it might be valuable for a Python script to be able to generate such outputs even if it can't read it.

A debatable feature but not a bug, granted.

I've re-classified this as a feature and not a bug as the current approach is stable

Classifying this entire issue as a feature does not make sense to me... The demoes I gave in in:

... both create very basic, valid .ini files that return different data - no matter which .ini parser you read them back with. This is basically the same as SQL injection which I don't think anyone would ever classify as a "feature". Also, there's no backwards-compatibility considerations in cases where something clearly never worked.

It's pretty common to turn everything upside and down and rewrite everything to fix a collection of bugs all at once (many developers love to write new code and prefer that sort of approach), yet that does not make each of those bugs a "feature".

As mentioned in #69909, it is possible to use configparser without serializing to an .ini file and parsing anything back. Probably not a common use case for "somethingparser" but still a valid use case. No parsing bug in such a use case and no need to change that either.

Sorry, I added type-bug but it appears that it was not considered as such

gh-65697: Improved error msg for configparser key validation (#135527)

pythongh-65697: Improved error msg for configparser key validation (p…

[3.14] gh-65697: Improved error msg for configparser key validation (G…

I am willing to send a PR that turns this new exception into a DeprecationWarning, if that's what you mean by "driving it".

My concern is less about the immediate need and about the implied need to follow up two years from now to clean up the warning and replace it with an exception.

Subclassing and overriding _validate_key_contents (or even assigning a dummy function on an instance) works to disable this, but clearly is not meant as an API (the name is underscore-prefixed).

Agreed. But it might serve as an unofficial API for the rare edge case if there were, for example, a comment or docstring in that method stating that subclasses might override to customize behavior. It's perfectly reasonable for a subclass to override a private method ("protected" in C parlance), and it's unlikely to be changed anyway, so any small bit of protection could be sufficient (if informal).

The naming is probably a long established historical artifact. It's normal for a package to expand scope beyond its original intent or namesake. It's pretty obvious to me that configparser supports creating config files and not just parsing them.

Yes, obviously. That's not an evolution, everyone has expected "configparser" to create files from the very beginning. This debate is not about "creating files"; that's an incomplete and misleading presentation. It's about creating invalid files that cannot be parsed back and about (much worse) creating files that can are parsed differently = a great recipe for vulnerabilities.

pythongh-65697: Improved error msg for configparser key validation (p…

Navigation Menu

Repository navigation

configparser accepts invalid keys and sections when writing #65697

Linked PRs

16 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

configparser accepts invalid keys and sections when writing #65697

Description

Linked PRs

Activity

The-Compiler commented on May 13, 2014

isidentical commented on Jan 11, 2020

marc-hb commented on Jan 15, 2025

jaraco commented on Jan 15, 2025

jaraco commented on Jan 15, 2025

marc-hb commented on Jan 16, 2025

16 remaining items

picnixz commented on Jun 14, 2025

jaraco commented on Jun 15, 2025

marc-hb commented on Jun 15, 2025

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions