Repository navigation
Provide a toml module in the standard library #84240
Description
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee = None
closed_at = None
created_at = <Date 2020-03-25.06:54:21.437>
labels = ['type-feature', 'library']
title = 'Provide a toml module in the standard library'
updated_at = <Date 2022-03-09.13:38:15.300>
user = 'https://github.com/mgorny'bugs.python.org fields:
activity = <Date 2022-03-09.13:38:15.300>
actor = 'miss-islington'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-03-25.06:54:21.437>
creator = 'mgorny'
dependencies = []
files = []
hgrepos = []
issue_num = 40059
keywords = ['patch']
message_count = 27.0
messages = ['364979', '364983', '365000', '373879', '387807', '387892', '401071', '401421', '406458', '406484', '406488', '406515', '406561', '408316', '408320', '408418', '408503', '408505', '408576', '409488', '409489', '409552', '410353', '414731', '414746', '414800', '414801']
nosy_count = 20.0
nosy_names = ['brett.cannon', 'christian.heimes', 'mcepl', 'njs', 'petr.viktorin', 'alex.gronholm', 'mgorny', 'dstufft', 'pradyunsg', 'eschwartz', 'miss-islington', 'VA', 'xtreak', 'skoslowski', 'erlendaasland', 'hauntsaninja', 'domdfcoding', 'dom1310df', 'hukkinj1', 'YakoYakoYokuYoku']
pr_nums = ['31498', '31784']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue40059'
versions = []
Activity
mgorny commentedon Mar 25, 2020
PEP-518 uses the TOML format to specify build system requirements. AFAIU this means that all new build systems will require a TOML parser. Could you consider adding one to the standard library to reduce the number of chicken-egg problems?
The referenced PEP states that 'pytoml TOML parser is ~300 lines of pure Python code', so I don't think integrating it would be a large maintenance cost.
[1] https://www.python.org/dev/peps/pep-0518/
tirkarthi commentedon Mar 25, 2020
Relevant python-dev discussion : https://mail.python.org/pipermail/python-dev/2019-May/157405.html . The format has still not reached 1.0. Issue to track the 1.0 release candidate : toml-lang/toml#698
brettcannon commentedon Mar 25, 2020
The plan is to start discussing adding a TOML parser once the spec reaches 1.0 (which we will know about as one of the pip contributors also manages TOML).
VA commentedon Jul 18, 2020
1.0.0-rc.1 is out by now: https://github.com/toml-lang/toml/blob/master/CHANGELOG.md
11 remaining items
tiran commentedon Dec 11, 2021
I just noticed that tomli has dropped support for Python 3.6. That's a road block for general adoption of the package in the Python ecosystem. Python 3.6 is the default Python interpreter in CentOS 8, C8S, RHEL 8, and Ubuntu 18.04 LTS. hukkin/tomli#134
hauntsaninja commentedon Dec 13, 2021
Given that this currently seems blocked on the broad question of "how should additions and removals to the stdlib be managed", I'd like to not focus too hard just yet on the specifics of tomli. I assume it's unlikely, but for all we know, the SC could determine that all newly included modules have to be written from scratch, following along the lines of recent additions like zoneinfo, graphlib and importlib.metadata.
There's a lot we could bikeshed or debate... e.g., it's not even clear to me what a toml package in the stdlib would be named, never mind what it means for an unreleased version of a commonly used third party toml package dropping support for an imminently EOL version of Python. I'm possibly totally out of line, but in my head, the process would look something like this:
Step 1: Hear back from the SC about criteria and considerations for new modules in the stdlib
Step 2: Determine whether a TOML library *in the abstract* is something that would meet the outlined criteria (potentially e.g. is this something we even want? is this something we can maintain?)
Step 3: Determine if we have an implementation (written from scratch, copied, or derived from something pre-existing) that would meet the outlined criteria
Step 4: Do all the rest of the work to meet the outlined criteria (potentially e.g. go through PEP process, create proposed impl, write a backport, bikeshed api and name, etc)
I guess I have the following questions:
Brett previously mentioned bringing it up with the author of tomli, as per Please consider pushing tomli into stdlib hukkin/tomli#141 they seem supportive
I see no mention of stdlib changes in https://github.com/python/steering-council and the discuss thread linked above seems to have petered out.
brettcannon commentedon Dec 14, 2021
It's already in pip, so I think it's already generally adopted 😉. https://github.com/pypa/pip/tree/main/src/pip/_vendor/tomli
Not officially, no. But I'm personally not going to bring it forward right now. If someone else wants to formulate a complete proposal for the SC on this then they are definitely welcome to! You will need to address where the code is coming from, why that code should be used, what's the API, etc.
The only reason the SC is mentioned here is there will be a discussion about how to maintain the stdlib, but it simply hasn't happened yet. You don't have to wait for it and asking for a TOML module might actually force the issue.
Nope, someone eventually has to have the time to make the proposal and manage the deluge of comments.
https://github.com/python/steering-council as you already pointed out through the issues and monthly summaries. Otherwise you just need to open an issue and ask. 😃
YakoYakoYokuYoku commentedon Dec 14, 2021
I've been working on a document detailing the API implementation (as a PEP draft) see https://gitlab.com/YakoYakoYokuYoku/pep-toml. Although I've yet to write the code I'll to move forward with it.
brettcannon commentedon Dec 14, 2021
I opened python/steering-council#92 for the SC to discuss stdlib additions in case I am not re-elected.
hauntsaninja commentedon Jan 2, 2022
Happy new year, potentially-toml-wanting friends!
I wrote up a draft of a proposal here: https://gist.github.com/hauntsaninja/9f136a5a60f63d8ca2cdfadb50edba44
I've passed it along to hukkin (author of tomli) for feedback. Once they reply, I'll send it to python-dev and the deluge of comments can get underway. If in the meantime anyone else has thoughts, I'd love to hear from you.
hauntsaninja commentedon Jan 2, 2022
pradyunsg kindly pointed me to an ongoing thread in Packaging discuss: https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/84
So seems like a PEP will be necessary. I'm happy to do legwork for that. (The PEP that I'll write differs from YakoYakoYokuYoku's json-esque draft in that it'll be based on tomli instead of a from-scratch implementation)
hauntsaninja commentedon Jan 3, 2022
We've started a PEP draft.
https://github.com/hauntsaninja/peps/blob/toml-pep/pep-9999.rst shows a rendered version.
hauntsaninja/peps#1 is a PR from the toml-pep branch to main, to help ease of review / discussion.
If you're following along and need to get your bearings straight, here are the recently active locations of discussion:
hauntsaninja commentedon Jan 11, 2022
This is now PEP-680: https://www.python.org/dev/peps/pep-0680/
Currently being discussed at: https://discuss.python.org/t/pep-680-tomllib-support-for-parsing-toml-in-the-standard-library/13040
encukou commentedon Mar 8, 2022
New changeset 591f675 by Taneli Hukkinen in branch 'main':
bpo-40059: Add tomllib (PEP-680) (GH-31498)
591f675
encukou commentedon Mar 8, 2022
The PR is merged and buildbots are green. Thank you to everyone who helped!
Now would be a good time to bikeshed wording in the documentation.
From the PR:
dom1310df commentedon Mar 9, 2022
When building Python from source (as of the latest GitHub commit) the tomllib directory doesn't actually get copied over to the install prefix.
It looks like an entry's needed in Makefile.pre.in under LIBSUBDIRS, along the lines of #13563 and #30311
miss-islington commentedon Mar 9, 2022
New changeset 23dcea5 by Dominic Davis-Foster in branch 'main':
bpo-40059: Fix installation of tomllib (GH-31784)
23dcea5