Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-96168: Add sqlite3 row factory how-to #99507

Merged
Merged
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
160 changes: 124 additions & 36 deletions Doc/library/sqlite3.rst
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,7 @@ inserted data and retrieved values from it in multiple ways.
* :ref:`sqlite3-adapters`
* :ref:`sqlite3-converters`
* :ref:`sqlite3-connection-context-manager`
* :ref:`sqlite3-howto-row-factory`

* :ref:`sqlite3-explanation` for in-depth background on transaction control.

Expand Down Expand Up @@ -1320,27 +1321,12 @@ Connection objects
a :class:`Cursor` object and the raw row results as a :class:`tuple`,
and returns a custom object representing an SQLite row.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

Example:

.. doctest::

>>> def dict_factory(cursor, row):
... col_names = [col[0] for col in cursor.description]
... return {key: value for key, value in zip(col_names, row)}
>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = dict_factory
>>> for row in con.execute("SELECT 1 AS a, 2 AS b"):
... print(row)
{'a': 1, 'b': 2}

If returning a tuple doesn't suffice and you want name-based access to
columns, you should consider setting :attr:`row_factory` to the
highly optimized :class:`sqlite3.Row` type. :class:`Row` provides both
index-based and case-insensitive name-based access to columns with almost no
memory overhead. It will probably be better than your own custom
dictionary-based approach or even a db_row based solution.
If returning a tuple doesn't suffice
and name-based access to columns is needed,
:attr:`row_factory` can be set to the
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
highly optimized :class:`sqlite3.Row` row factory.

.. XXX what's a db_row-based solution?
See :ref:`sqlite3-howto-row-factory` for more details.

.. attribute:: text_factory

Expand Down Expand Up @@ -1591,6 +1577,15 @@ Cursor objects
including :abbr:`CTE (Common Table Expression)` queries.
It is only updated by the :meth:`execute` and :meth:`executemany` methods.

.. attribute:: row_factory

This attribute is copied from :attr:`Connection.row_factory`
upon :class:`!Cursor` creation.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
Assigning to this attribute does not affect the original
connection :attr:`!row_factory`.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

See :ref:`sqlite3-howto-row-factory` for more details.


.. The sqlite3.Row example used to be a how-to. It has now been incorporated
into the Row reference. We keep the anchor here in order not to break
Expand All @@ -1609,7 +1604,10 @@ Row objects
It supports iteration, equality testing, :func:`len`,
and :term:`mapping` access by column name and index.

Two row objects compare equal if have equal columns and equal members.
Two :class:`!Row` objects compare equal
if they have equal columns and equal members.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

See :ref:`sqlite3-howto-row-factory` for more details.

.. method:: keys

Expand All @@ -1620,21 +1618,6 @@ Row objects
.. versionchanged:: 3.5
Added support of slicing.

Example:

.. doctest::

>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = sqlite3.Row
>>> res = con.execute("SELECT 'Earth' AS name, 6378 AS radius")
>>> row = res.fetchone()
>>> row.keys()
['name', 'radius']
>>> row[0], row["name"] # Access by index and name.
('Earth', 'Earth')
>>> row["RADIUS"] # Column names are case-insensitive.
6378


.. _sqlite3-blob-objects:

Expand Down Expand Up @@ -2358,6 +2341,111 @@ can be found in the `SQLite URI documentation`_.
.. _SQLite URI documentation: https://www.sqlite.org/uri.html


.. _sqlite3-howto-row-factory:

How to create and use row factories
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

By default, :mod:`!sqlite3` represents each fetched row as a :class:`tuple`.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
If a :class:`!tuple` does not suit your needs,
use the :class:`sqlite3.Row` class or a custom :attr:`~Connection.row_factory`.

:class:`!Row` provides indexed and case-insensitive named access to columns,
with low memory overhead and minimal performance impact.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
In order to use :class:`!Row` as a row factory,
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
simply assign it to the :attr:`Connection.row_factory` attribute:
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

.. doctest::

>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = sqlite3.Row

Query results are now returned as :class:`!Row` instances:
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

.. doctest::

>>> res = con.execute("SELECT 'Earth' AS name, 6378 AS radius")
>>> row = res.fetchone()
>>> row.keys()
['name', 'radius']
>>> row[0] # Access by index.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
'Earth'
>>> row["name"] # Access by name.
'Earth'
>>> row["RADIUS"] # Column names are case-insensitive.
6378

It is also possible to assign row factories to cursors using
:attr:`Cursor.row_factory`:

.. doctest::

>>> cur = con.cursor()
>>> cur.row_factory == con.row_factory
True
>>> cur.row_factory = None # Override cursor row factory.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

# The cursor and the connection row factories now differ.
>>> cur.row_factory == con.row_factory
False
>>> cur.execute("SELECT 'Hello'").fetchone()
('Hello',)

# The connection still uses sqlite3.Row, as set in the previous example.
>>> con.execute("SELECT 'Hello'").fetchone()
<sqlite3.Row object at ...>

If more flexibility is needed, implement a custom row factory.
Here's an example of one returning a :class:`dict`:
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

.. doctest::
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

>>> def dict_factory(cursor, row):
... col_names = [col[0] for col in cursor.description]
... return {key: value for key, value in zip(col_names, row)}

>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = dict_factory
>>> for row in con.execute("SELECT 1 AS a, 2 AS b"):
... print(row)
{'a': 1, 'b': 2}

The following row factory returns a :class:`~collections.namedtuple`.
merwok marked this conversation as resolved.
Show resolved Hide resolved

.. testcode::

from collections import namedtuple

def namedtuple_factory(cursor, row):
fields = [col[0] for col in cursor.description]
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
cls = namedtuple("Row", fields)
return cls._make(row)
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved

:func:`!namedtuple_factory` can be used as follows:

.. doctest::

>>> con = sqlite3.connect(":memory:")
>>> con.row_factory = namedtuple_factory
>>> cur = con.execute("SELECT 1 AS a, 2 AS b")
>>> row = cur.fetchone()
>>> row
Row(a=1, b=2)
>>> row[0] # Indexed access.
1
>>> row.b # Attribute access.
2

With some adjustments, the above recipe can be adapted to use a
:class:`~dataclasses.dataclass`, or any other custom class,
instead of a :class:`~collections.namedtuple`.

As an exercise left for the reader, the above example can be optimised
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved
by extracting to a new function the code that create the ``fields`` variable.
Then decorate that new function with :func:`functools.lru_cache` to avoid
creating multiple classes ``cls`` from identical column specs.
erlend-aasland marked this conversation as resolved.
Show resolved Hide resolved


.. _sqlite3-explanation:

Explanation
Expand Down