Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PEP 680: link discuss post, minor edits #2226

Merged
merged 7 commits into from
Jan 12, 2022
Merged

PEP 680: link discuss post, minor edits #2226

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3680164
PEP 680: link discuss post, minor edits
Jan 11, 2022
66b02ea
update CODEOWNERS
Jan 11, 2022
e03b3ef
Apply suggestions from code review
hauntsaninja Jan 11, 2022
9af0af5
Review feedback
Jan 11, 2022
7b2f3f0
Cite reasons for "significant fraction"
Jan 12, 2022
490fdb7
empty commit to restart ci
Jan 12, 2022
bb96494
Fix link
Jan 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
View file
Open in desktop
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -535,6 +535,7 @@ pep-0676.rst @Mariatta
pep-0677.rst @gvanrossum
pep-0678.rst @iritkatriel
pep-0679.rst @pablogsal
pep-0680.rst @encukou
# ...
# pep-0754.txt
# ...
Expand Down
71 changes: 32 additions & 39 deletions pep-0680.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
PEP: 680
Title: tomllib: Support for parsing TOML in the Standard Library
Title: tomllib: Support for Parsing TOML in the Standard Library
Author: Taneli Hukkinen, Shantanu Jain <hauntsaninja at gmail.com>
hauntsaninja marked this conversation as resolved.
Show resolved Hide resolved
Sponsor: Petr Viktorin <[email protected]>
Discussions-To: https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068
Discussions-To: https://discuss.python.org/t/13040
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 01-Jan-2022
Python-Version: 3.11
Post-History: 1900-01-01
Post-History: 11-Jan-2022


Abstract
Expand Down Expand Up @@ -189,35 +189,31 @@ Rejected Ideas
Basing on another TOML implementation
-------------------------------------

Potential alternatives include:
Several potential alternative implementations exist:

* ``tomlkit``.
``tomlkit`` is well established, actively maintained and supports TOML v1. An
* ``tomlkit`` is well established, actively maintained and supports TOML v1. An
important difference is that ``tomlkit`` supports style roundtripping. As a
result, it has a more complex API and implementation (about 5x as much code as
``tomli``). The author does not believe that ``tomlkit`` is a good choice for
``tomli``). Its author does not believe that ``tomlkit`` is a good choice for
the standard library.

* ``toml``.
``toml`` is a widely used library. However, it is not actively maintained,
does not support TOML v1 and has several known bugs. Its API is more complex
than that of ``tomli``. It has some very limited and mostly unused ability to
preserve style through an undocumented decoder API. It has the ability to
customise output style through a complicated encoder API. For more details on
API differences to this PEP, refer to `Appendix A`_.
* ``toml`` is a very widely used library. However, it is not actively
maintained, does not support TOML v1 and has several known bugs. Its API is
more complex than that of ``tomli``. It has some very limited and mostly
unused ability to preserve style through an undocumented decoder API. It has
the ability to customise output style through a complicated encoder API. For
more details on API differences to this PEP, refer to `Appendix A`_.

* ``pytomlpp``.
``pytomlpp`` is a Python wrapper for the C++ project ``toml++``. Pure Python
* ``pytomlpp`` is a Python wrapper for the C++ project ``toml++``. Pure Python
libraries are easier to maintain than extension modules.

* ``rtoml``.
``rtoml`` is a Python wrapper for the Rust project ``toml-rs`` and hence has
* ``rtoml`` is a Python wrapper for the Rust project ``toml-rs`` and hence has
similar shortcomings to ``pytomlpp``.
In addition, it does not support TOML v1.

* Writing from scratch.
It's unclear what we would get from this: ``tomli`` meets our needs and the
author is willing to help with its inclusion in the standard library.
* Writing an implementation from scratch. It's unclear what we would get from
this: ``tomli`` meets our needs and the author is willing to help with its
inclusion in the standard library.

Including an API for writing TOML
---------------------------------
Expand Down Expand Up @@ -277,7 +273,7 @@ a text string. This may seem inconsistent at first.

Quoting TOML v1.0.0 specification:

> A TOML file must be a valid UTF-8 encoded Unicode document.
A TOML file must be a valid UTF-8 encoded Unicode document.

``tomllib.loads`` does not intend to load a TOML file, but rather the
document that the file stores. The most natural representation of
Expand Down Expand Up @@ -342,9 +338,9 @@ Finally, the ``toml`` package on PyPI is not actively maintained and `we have
been unable to contact the author <https://github.com/uiri/toml/issues/361>`__,
so action here would likely have to be taken without the author's consent.

This PEP proposes ``tomllib``. This mirrors ``plistlib`` (another file format
module in the standard library), as well as several others such as ``pathlib``,
``graphlib``, etc.
This PEP proposes ``tomllib``. This mirrors ``plistlib`` and ``xdrlib`` (two
other file format modules in the standard library), as well as several others
such as ``pathlib``, ``contextlib``, ``graphlib``, etc.

Other considered names include:

Expand All @@ -356,22 +352,14 @@ Other considered names include:
``marshal``, ``html`` etc. would not be included in the namespace.


TODO: Random things
Previous Discussion
===================

Previous discussion:

* https://bugs.python.org/issue40059
* https://mail.python.org/archives/list/[email protected]/thread/IWJ3I32A4TY6CIVQ6ONPEBPWP4TOV2V7/
* https://mail.python.org/pipermail/python-dev/2019-May/157405.html
* https://github.com/hukkin/tomli/issues/141
* https://mail.python.org/archives/list/[email protected]/thread/IWJ3I32A4TY6CIVQ6ONPEBPWP4TOV2V7/
* https://discuss.python.org/t/adopting-recommending-a-toml-parser/4068/84

Useful https://grep.app searches (note, ignore vendored):

* toml.load[s] usage https://grep.app/search?q=toml.load&filter[lang][0]=Python
* toml.dump[s] usage https://grep.app/search?q=toml.dump&filter[lang][0]=Python
* TomlEncoder subclasses https://grep.app/search?q=TomlEncoder%29%3A&filter[lang][0]=Python
hauntsaninja marked this conversation as resolved.
Show resolved Hide resolved
* https://github.com/hukkin/tomli/issues/141


.. _Appendix A:
Expand All @@ -394,7 +382,9 @@ space. Note that this list might not be exhaustive.
code that uses ``toml`` to use the API proposed in this PEP (acknowledging
that that is very different from a compatible API).

A significant fraction of ``toml`` users rely on this.
A significant fraction of ``toml`` users rely on this, based on comparing
`occurrences of "toml.load" <https://grep.app/search?q=toml.load&filter[lang][0]=Python>`__
to `occurences of "toml.dump" <https://grep.app/search?q=toml.dump&filter[lang][0]=Python>`__.

#. Different first argument of ``toml.load``

Expand Down Expand Up @@ -422,14 +412,17 @@ space. Note that this list might not be exhaustive.
sequences, and hence is simply stricter format than what text file objects
represent.

A significant fraction of ``toml`` users rely on this.
A significant fraction of ``toml`` users rely on this, based on manual inspection
of `occurrences of "toml.load" <https://grep.app/search?q=toml.load&filter[lang][0]=Python>`__.

#. Errors

``toml`` raises ``TomlDecodeError`` vs the proposed PEP 8 compliant
``TOMLDecodeError``.

A significant fraction of ``toml`` users rely on this.
A significant fraction of ``toml`` users rely on this, based on `occurrences
of "TomlDecodeError"
<https://grep.app/search?q=TomlDecodeError&case=true&filter[lang][0]=Python>`__.

#. ``toml.load[s]`` accepts a ``_dict`` argument

Expand Down