Merge branch 'main' into sqlite-autocommit

.github/CODEOWNERS

@@ -18,7 +18,7 @@ Objects/codeobject.c          @markshannon
 Objects/frameobject.c         @markshannon
 Objects/call.c                @markshannon
 Python/ceval.c                @markshannon
-Python/compile.c              @markshannon
+Python/compile.c              @markshannon @iritkatriel
 Python/ast_opt.c              @isidentical
 Lib/test/test_patma.py        @brandtbucher
 Lib/test/test_peepholer.py    @brandtbucher
@@ -29,7 +29,6 @@ Lib/test/test_except*.py      @iritkatriel
 Lib/test/test_traceback.py    @iritkatriel
 Objects/exceptions.c          @iritkatriel
 Python/traceback.c            @iritkatriel
-Python/pythonrun.c            @iritkatriel
 
 # Hashing
 **/*hashlib*                  @tiran

Doc/c-api/intro.rst

@@ -153,7 +153,7 @@ complete listing.
 .. c:macro:: Py_GETENV(s)
 
    Like ``getenv(s)``, but returns ``NULL`` if :option:`-E` was passed on the
-   command line (i.e. if ``Py_IgnoreEnvironmentFlag`` is set).
+   command line (see :c:member:`PyConfig.use_environment`).
 
 .. c:macro:: Py_MAX(x, y)
 

Doc/c-api/sys.rst

@@ -167,7 +167,7 @@ Operating System Utilities
 
    .. versionchanged:: 3.8
       The function now uses the UTF-8 encoding on Windows if
-      :c:data:`Py_LegacyWindowsFSEncodingFlag` is zero;
+      :c:member:`PyConfig.legacy_windows_fs_encoding` is zero;
 
 
 .. c:function:: char* Py_EncodeLocale(const wchar_t *text, size_t *error_pos)
@@ -209,7 +209,7 @@ Operating System Utilities
 
    .. versionchanged:: 3.8
       The function now uses the UTF-8 encoding on Windows if
-      :c:data:`Py_LegacyWindowsFSEncodingFlag` is zero.
+      :c:member:`PyConfig.legacy_windows_fs_encoding` is zero.
 
 
 .. _systemfunctions:

Doc/c-api/veryhigh.rst

@@ -39,7 +39,7 @@ the same library that the Python runtime is using.
 
    Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
    function will not return ``1``, but exit the process, as long as
-   ``Py_InspectFlag`` is not set.
+   :c:member:`PyConfig.inspect` is zero.
 
 
 .. c:function:: int Py_BytesMain(int argc, char **argv)
@@ -95,7 +95,7 @@ the same library that the Python runtime is using.
 
    Note that if an otherwise unhandled :exc:`SystemExit` is raised, this
    function will not return ``-1``, but exit the process, as long as
-   ``Py_InspectFlag`` is not set.
+   :c:member:`PyConfig.inspect` is zero.
 
 
 .. c:function:: int PyRun_SimpleFile(FILE *fp, const char *filename)

Doc/library/ctypes.rst

@@ -1068,18 +1068,16 @@ Accessing values exported from dlls
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
 Some shared libraries not only export functions, they also export variables. An
-example in the Python library itself is the :c:data:`Py_OptimizeFlag`, an integer
-set to 0, 1, or 2, depending on the :option:`-O` or :option:`-OO` flag given on
-startup.
+example in the Python library itself is the :c:data:`Py_Version`, Python
+runtime version number encoded in a single constant integer.
 
 :mod:`ctypes` can access values like this with the :meth:`in_dll` class methods of
 the type.  *pythonapi* is a predefined symbol giving access to the Python C
 api::
 
-   >>> opt_flag = c_int.in_dll(pythonapi, "Py_OptimizeFlag")
-   >>> print(opt_flag)
-   c_long(0)
-   >>>
+   >>> version = ctypes.c_int.in_dll(ctypes.pythonapi, "Py_Version")
+   >>> print(hex(version.value))
+   0x30c00a0
 
 If the interpreter would have been started with :option:`-O`, the sample would
 have printed ``c_long(1)``, or ``c_long(2)`` if :option:`-OO` would have been

Doc/library/errno.rst

@@ -657,3 +657,12 @@ defined by the module.  The specific list of defined symbols is available as
    Interface output queue is full
 
    .. versionadded:: 3.11
+
+.. data:: ENOTCAPABLE
+
+   Capabilities insufficient. This error is mapped to the exception
+   :exc:`PermissionError`.
+
+   .. availability:: WASI, FreeBSD
+
+   .. versionadded:: 3.11.1

Doc/library/exceptions.rst

@@ -746,7 +746,12 @@ depending on the system error code.
 
    Raised when trying to run an operation without the adequate access
    rights - for example filesystem permissions.
-   Corresponds to :c:data:`errno` :py:data:`~errno.EACCES` and :py:data:`~errno.EPERM`.
+   Corresponds to :c:data:`errno` :py:data:`~errno.EACCES`,
+   :py:data:`~errno.EPERM`, and :py:data:`~errno.ENOTCAPABLE`.
+
+   .. versionchanged:: 3.11.1
+      WASI's :py:data:`~errno.ENOTCAPABLE` is now mapped to
+      :exc:`PermissionError`.
 
 .. exception:: ProcessLookupError
 

Doc/library/sqlite3.rst

@@ -47,85 +47,173 @@ This document includes four main sections:
       PEP written by Marc-André Lemburg.
 
 
+.. We use the following practises for SQL code:
+   - UPPERCASE for keywords
+   - snake_case for schema
+   - single quotes for string literals
+   - singular for table names
+   - if needed, use double quotes for table and column names
+
 .. _sqlite3-tutorial:
 
 Tutorial
 --------
 
-To use the module, start by creating a :class:`Connection` object that
-represents the database.  Here the data will be stored in the
-:file:`example.db` file::
+In this tutorial, you will create a database of Monty Python movies
+using basic :mod:`!sqlite3` functionality.
+It assumes a fundamental understanding of database concepts,
+including `cursors`_ and `transactions`_.
+
+First, we need to create a new database and open
+a database connection to allow :mod:`!sqlite3` to work with it.
+Call :func:`sqlite3.connect` to to create a connection to
+the database :file:`tutorial.db` in the current working directory,
+implicitly creating it if it does not exist::
 
    import sqlite3
-   con = sqlite3.connect('example.db')
+   con = sqlite3.connect("tutorial.db")
 
-The special path name ``:memory:`` can be provided to create a temporary
-database in RAM.
+The returned :class:`Connection` object ``con``
+represents the connection to the on-disk database.
 
-Once a :class:`Connection` has been established, create a :class:`Cursor` object
-and call its :meth:`~Cursor.execute` method to perform SQL commands::
+In order to execute SQL statements and fetch results from SQL queries,
+we will need to use a database cursor.
+Call :meth:`con.cursor() <Connection.cursor>` to create the :class:`Cursor`::
 
    cur = con.cursor()
 
-   # Create table
-   cur.execute('''CREATE TABLE stocks
-                  (date text, trans text, symbol text, qty real, price real)''')
-
-   # Insert a row of data
-   cur.execute("INSERT INTO stocks VALUES ('2006-01-05','BUY','RHAT',100,35.14)")
+Now that we've got a database connection and a cursor,
+we can create a database table ``movie`` with columns for title,
+release year, and review score.
+For simplicity, we can just use column names in the table declaration --
+thanks to the `flexible typing`_ feature of SQLite,
+specifying the data types is optional.
+Execute the ``CREATE TABLE`` statement
+by calling :meth:`cur.execute(...) <Cursor.execute>`::
+
+   cur.execute("CREATE TABLE movie(title, year, score)")
+
+.. Ideally, we'd use sqlite_schema instead of sqlite_master below,
+   but SQLite versions older than 3.33.0 do not recognise that variant.
+
+We can verify that the new table has been created by querying
+the ``sqlite_master`` table built-in to SQLite,
+which should now contain an entry for the ``movie`` table definition
+(see `The Schema Table`_ for details).
+Execute that query by calling :meth:`cur.execute(...) <Cursor.execute>`,
+assign the result to ``res``,
+and call :meth:`res.fetchone() <Cursor.fetchone>` to fetch the resulting row::
+
+   >>> res = cur.execute("SELECT name FROM sqlite_master")
+   >>> res.fetchone()
+   ('movie',)
+
+We can see that the table has been created,
+as the query returns a :class:`tuple` containing the table's name.
+If we query ``sqlite_master`` for a non-existent table ``spam``,
+:meth:`!res.fetchone()` will return ``None``::
+
+   >>> res = cur.execute("SELECT name FROM sqlite_master WHERE name='spam'")
+   >>> res.fetchone() is None
+   True
+
+Now, add two rows of data supplied as SQL literals
+by executing an ``INSERT`` statement,
+once again by calling :meth:`cur.execute(...) <Cursor.execute>`::
+
+   cur.execute("""
+       INSERT INTO movie VALUES
+           ('Monty Python and the Holy Grail', 1975, 8.2),
+           ('And Now for Something Completely Different', 1971, 7.5)
+   """)
+
+The ``INSERT`` statement implicitly opens a transaction,
+which needs to be committed before changes are saved in the database
+(see :ref:`sqlite3-controlling-transactions` for details).
+Call :meth:`con.commit() <Connection.commit>` on the connection object
+to commit the transaction::
 
-   # Save (commit) the changes
    con.commit()
 
-   # We can also close the connection if we are done with it.
-   # Just be sure any changes have been committed or they will be lost.
-   con.close()
-
-The saved data is persistent: it can be reloaded in a subsequent session even
-after restarting the Python interpreter::
-
-   import sqlite3
-   con = sqlite3.connect('example.db')
-   cur = con.cursor()
+We can verify that the data was inserted correctly
+by executing a ``SELECT`` query.
+Use the now-familiar :meth:`cur.execute(...) <Cursor.execute>` to
+assign the result to ``res``,
+and call :meth:`res.fetchall() <Cursor.fetchall>` to return all resulting rows::
 
-At this point, our database only contains one row::
+   >>> res = cur.execute("SELECT score FROM movie")
+   >>> res.fetchall()
+   [(8.2,), (7.5,)]
 
-   >>> res = cur.execute('SELECT count(rowid) FROM stocks')
-   >>> print(res.fetchone())
-   (1,)
+The result is a :class:`list` of two :class:`!tuple`\s, one per row,
+each containing that row's ``score`` value.
 
-The result is a one-item :class:`tuple`:
-one row, with one column.
-Now, let us insert three more rows of data,
-using :meth:`~Cursor.executemany`::
+Now, insert three more rows by calling
+:meth:`cur.executemany(...) <Cursor.executemany>`::
 
-   >>> data = [
-   ...    ('2006-03-28', 'BUY', 'IBM', 1000, 45.0),
-   ...    ('2006-04-05', 'BUY', 'MSFT', 1000, 72.0),
-   ...    ('2006-04-06', 'SELL', 'IBM', 500, 53.0),
-   ... ]
-   >>> cur.executemany('INSERT INTO stocks VALUES(?, ?, ?, ?, ?)', data)
+   data = [
+       ("Monty Python Live at the Hollywood Bowl", 1982, 7.9),
+       ("Monty Python's The Meaning of Life", 1983, 7.5),
+       ("Monty Python's Life of Brian", 1979, 8.0),
+   ]
+   cur.executemany("INSERT INTO movie VALUES(?, ?, ?)", data)
+   con.commit()  # Remember to commit the transaction after executing INSERT.
 
-Notice that we used ``?`` placeholders to bind *data* to the query.
+Notice that ``?`` placeholders are used to bind ``data`` to the query.
 Always use placeholders instead of :ref:`string formatting <tut-formatting>`
 to bind Python values to SQL statements,
-to avoid `SQL injection attacks`_.
-See the :ref:`placeholders how-to <sqlite3-placeholders>` for more details.
+to avoid `SQL injection attacks`_
+(see :ref:`sqlite3-placeholders` for more details).
 
-Then, retrieve the data by iterating over the result of a ``SELECT`` statement::
+We can verify that the new rows were inserted
+by executing a ``SELECT`` query,
+this time iterating over the results of the query::
 
-   >>> for row in cur.execute('SELECT * FROM stocks ORDER BY price'):
+   >>> for row in cur.execute("SELECT year, title FROM movie ORDER BY year"):
    ...     print(row)
+   (1971, "And Now for Something Completely Different")
+   (1975, "Monty Python and the Holy Grail")
+   (1979, "Monty Python's Life of Brian")
+   (1982, "Monty Python Live at the Hollywood Bowl")
+   (1983, "Monty Python's The Meaning of Life")
+
+Each row is a two-item :class:`tuple` of ``(year, title)``,
+matching the columns selected in the query.
+
+Finally, verify that the database has been written to disk
+by calling :meth:`con.close() <Connection.close>`
+to close the existing connection, opening a new one,
+creating a new cursor, then querying the database::
+
+   >>> con.close()
+   >>> new_con = sqlite3.connect("tutorial.db")
+   >>> new_cur = new_con.cursor()
+   >>> res = new_cur.execute("SELECT year, title FROM movie ORDER BY score DESC"):
+   >>> title, year = res.fetchone()
+   >>> print(f'The highest scoring Monty Python movie is {title!r}, released in {year}')
+   'The highest scoring Monty Python movie is "Monty Python and the Holy Grail", released in 1975'
+
+You've now created an SQLite database using the :mod:`!sqlite3` module,
+inserted data and retrieved values from it in multiple ways.
+
+.. _SQL injection attacks: https://en.wikipedia.org/wiki/SQL_injection
+.. _The Schema Table: https://www.sqlite.org/schematab.html
+.. _cursors: https://en.wikipedia.org/wiki/Cursor_(databases)
+.. _flexible typing: https://www.sqlite.org/flextypegood.html
+.. _sqlite_master: https://www.sqlite.org/schematab.html
+.. _transactions: https://en.wikipedia.org/wiki/Database_transaction
 
-   ('2006-01-05', 'BUY', 'RHAT', 100, 35.14)
-   ('2006-03-28', 'BUY', 'IBM', 1000, 45.0)
-   ('2006-04-06', 'SELL', 'IBM', 500, 53.0)
-   ('2006-04-05', 'BUY', 'MSFT', 1000, 72.0)
+.. seealso::
 
-You've now created an SQLite database using the :mod:`!sqlite3` module.
+   * :ref:`sqlite3-howtos` for further reading:
 
-.. _SQL injection attacks: https://en.wikipedia.org/wiki/SQL_injection
+      * :ref:`sqlite3-placeholders`
+      * :ref:`sqlite3-adapters`
+      * :ref:`sqlite3-converters`
+      * :ref:`sqlite3-columns-by-name`
+      * :ref:`sqlite3-connection-context-manager`
 
+   * :ref:`sqlite3-explanation` for in-depth background on transaction control.
 
 .. _sqlite3-reference:
 
@@ -525,7 +613,7 @@ Connection objects
       supplied, this must be a callable returning an instance of :class:`Cursor`
       or its subclasses.
 
-   .. method:: blobopen(table, column, row, /, *, readonly=False, name="main")
+   .. method:: blobopen(table, column, row, /, \*, readonly=False, name="main")
 
       Open a :class:`Blob` handle to an existing
       :abbr:`BLOB (Binary Large OBject)`.
@@ -603,7 +691,7 @@ Connection objects
       :meth:`~Cursor.executescript` on it with the given *sql_script*.
       Return the new cursor object.
 
-   .. method:: create_function(name, narg, func, *, deterministic=False)
+   .. method:: create_function(name, narg, func, \*, deterministic=False)
 
       Create or remove a user-defined SQL function.
 
@@ -894,7 +982,7 @@ Connection objects
          con.close()
 
 
-   .. method:: backup(target, *, pages=-1, progress=None, name="main", sleep=0.250)
+   .. method:: backup(target, \*, pages=-1, progress=None, name="main", sleep=0.250)
 
       Create a backup of an SQLite database.
 
@@ -1006,7 +1094,7 @@ Connection objects
    .. _SQLite limit category: https://www.sqlite.org/c3ref/c_limit_attached.html
 
 
-   .. method:: serialize(*, name="main")
+   .. method:: serialize(\*, name="main")
 
       Serialize a database into a :class:`bytes` object.  For an
       ordinary on-disk database file, the serialization is just a copy of the
@@ -1028,7 +1116,7 @@ Connection objects
       .. versionadded:: 3.11
 
 
-   .. method:: deserialize(data, /, *, name="main")
+   .. method:: deserialize(data, /, \*, name="main")
 
       Deserialize a :meth:`serialized <serialize>` database into a
       :class:`Connection`.