Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QA: mypy, reformatting, and linting #69

Merged
merged 5 commits into from
Jun 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@ jobs:
- name: test
run: make test INSTALL_EXTRA=test

# lint:
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v3
# - uses: actions/setup-python@v4
# with:
# python-version: "3.10"
# - name: lint
# run: make lint INSTALL_EXTRA=lint
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v4
with:
python-version: "3.x"
- name: lint
run: make lint INSTALL_EXTRA=lint
42 changes: 42 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
on:
release:
types:
- published

name: release

jobs:
pypi:
name: upload release to PyPI
runs-on: ubuntu-latest
environment: release
permissions:
# Used for OIDC publishing.
# Used to sign the release's artifacts with sigstore-python.
id-token: write

# Used to attach signing artifacts to the published release.
contents: write

steps:
- uses: actions/checkout@v3

- uses: actions/setup-python@v4
with:
python-version: "3.x"

- name: deps
run: python -m pip install -U setuptools build wheel

- name: build
run: python -m build

- name: publish
uses: pypa/[email protected]

- name: sign
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not familiar with this step yet, but we sign after we've published to PyPI?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, for the moment -- PyPI doesn't support these signatures (yet) and the action writes to the same input directory, so doing it before the publishing step confuses the bejeezus out of the underlying twine upload dist/* invocation.

Happy to refactor, though, if you'd prefer -- this is primarily laziness on my part (and it's what I do on other projects for the time being) 🙂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need for a refactor - I was curious as to the order of operations.

However, now that you've mentioned twine, it makes me inquire "why use flit at all" if the gh-action responsbile for the publish part is already using twine, and flit's primary purpose is to:

Flit is a simple way to put Python packages and modules on PyPI.

If we're not using it for its primary function, should we continue to use it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been using it for its PEP 517 build backend, out of muscle memory -- IIRC I chose it years ago because it had a mature 517 backend at a time when setuptools' own backend was still a work in progress.

(I actually had no idea it also supported publishing -- I thought its primary function was to be a packaging backend a la setuptools or hatch.)

IMO we should continue to use it unless we have a compelling security or performance reason not to -- switching to another 517 backend would involve a small amount of pyproject.toml churn for no discernible change in output 🙂

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually had no idea [Fl]it also supported publishing -- I thought its primary function was to be a packaging backend a la setuptools or hatch.

Nowadays that's probably its main function, but it predates PEP 517, and I wanted to be able to build packages and publish them with one command. And so far the upload functionality hasn't been all that much effort to maintain (although I'm just now working on better support for using tokens to do uploads).

uses: sigstore/[email protected]
with:
inputs: ./dist/*.tar.gz ./dist/*.whl
release-signing-artifacts: true
bundle-only: true
5 changes: 2 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,7 @@
# stdlib-list

This package includes lists of all of the standard libraries for Python 2.6,
2.7, 3.2, 3.3, 3.4, 3.5, 3.6, 3.7, 3.8, and 3.9 along with the code for
scraping the official Python docs to get said lists.
This package includes lists of all of the standard libraries for Python 2.6
through 3.11.

**IMPORTANT**: If you're on Python 3.10 or newer, you **probably don't need this library**.
See [`sys.stdlib_module_names`](https://docs.python.org/3/library/sys.html#sys.stdlib_module_names)
Expand Down
7 changes: 3 additions & 4 deletions docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,9 @@
extensions = [
"sphinx.ext.autodoc",
"sphinx.ext.viewcode",
"sphinx_rtd_theme",
]

html_theme = "sphinx_rtd_theme"
html_theme = "furo"

# Add any paths that contain templates here, relative to this directory.
templates_path = ["_templates"]
Expand All @@ -48,8 +47,8 @@
master_doc = "index"

# General information about the project.
project = "Python Standard Library List"
copyright = "2015, Jack Maney"
project = "stdlib-list"
copyright = "2015, stdlib-list authors"

# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
Expand Down
24 changes: 9 additions & 15 deletions docs/index.rst
Original file line number Diff line number Diff line change
@@ -1,21 +1,15 @@
.. Python Standard Library List documentation master file, created by
sphinx-quickstart on Tue Mar 10 02:16:08 2015.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
stdlib-list
===========

Python Standard Library List
========================================================
This package includes lists of all of the standard libraries for Python 2.6
through 3.11.

This package includes lists of all of the standard libraries for Python 2.6, 2.7, 3.2, 3.3, and 3.4, along with the code for scraping the official Python docs to get said lists.
.. note::

Listing the modules in the standard library? Wait, why on Earth would you care about that?!
===========================================================================================

Because knowing whether or not a module is part of the standard library will come in handy in `a project of mine <https://github.com/jackmaney/pypt>`_. `And I'm not the only one <http://stackoverflow.com/questions/6463918/how-can-i-get-a-list-of-all-the-python-standard-library-modules>`_ who would find this useful. Or, the TL;DR answer is that it's handy in situations when you're analyzing Python code and would like to find module dependencies.

After googling for a way to generate a list of Python standard libraries (and looking through the answers to the previously-linked Stack Overflow question), I decided that I didn't like the existing solutions. So, I started by writing a scraper for the TOC of the Python Module Index for each of the versions of Python above.

However, web scraping can be a fragile affair. Thanks to `a suggestion <https://github.com/jackmaney/python-stdlib-list/issues/1#issuecomment-86517208>`_ by `@ncoghlan <https://github.com/ncoghlan>`_, and some further help from `@birkenfeld <https://github.com/birkenfeld>`_ and `@epc <https://github.com/epc>`_, the population of the lists is now done by grabbing and parsing the Sphinx object inventory for the official Python docs of each relevant version.
If you're on Python 3.10 or newer, you **probably don't need this library**.
See `sys.stdlib_module_names <https://docs.python.org/3/library/sys.html#sys.stdlib_module_names>`_
and `sys.builtin_module_names <https://docs.python.org/3/library/sys.html#sys.builtin_module_names>`_
for similar functionality.

Contents
========
Expand Down
19 changes: 18 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -30,15 +30,32 @@ Source = "https://github.com/pypi/stdlib-list"
[project.optional-dependencies]
test = ["pytest", "pytest-cov", "coverage[toml]"]
lint = ["black", "mypy", "ruff"]
doc = ["sphinx", "sphinx_rtd_theme"]
doc = ["sphinx", "furo"]
dev = ["build", "stdlib-list[test,lint,doc]"]
# CI only: used for list generation for Python versions < 3.10.
support = ["sphobjinv"]

[tool.mypy]
allow_redefinition = true
check_untyped_defs = true
disallow_incomplete_defs = true
disallow_untyped_defs = true
ignore_missing_imports = true
no_implicit_optional = true
show_error_codes = true
sqlite_cache = true
strict_equality = true
warn_no_return = true
warn_redundant_casts = true
warn_return_any = true
warn_unreachable = true
warn_unused_configs = true
warn_unused_ignores = true

[tool.black]
line-length = 100

[tool.ruff]
select = ["E", "F", "I", "W", "UP"]
target-version = "py37"
line-length = 100
6 changes: 3 additions & 3 deletions stdlib_list/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@

# Import all the things that used to be in here for backwards-compatibility reasons
from .base import (
stdlib_list,
in_stdlib,
get_canonical_version,
short_versions,
in_stdlib,
long_versions,
short_versions,
stdlib_list,
)

__all__ = [
Expand Down
17 changes: 8 additions & 9 deletions stdlib_list/base.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,8 @@
from __future__ import print_function, absolute_import
from __future__ import annotations

import os
import pkgutil
import sys

from functools import lru_cache

long_versions = [
Expand All @@ -24,16 +23,16 @@
short_versions = [".".join(x.split(".")[:2]) for x in long_versions]


def get_canonical_version(version):
def get_canonical_version(version: str) -> str:
if version in long_versions:
version = ".".join(version.split(".")[:2])
elif version not in short_versions:
raise ValueError("No such version: {}".format(version))
raise ValueError(f"No such version: {version}")

return version


def stdlib_list(version=None):
def stdlib_list(version: str | None = None) -> list[str]:
"""
Given a ``version``, return a ``list`` of names of the Python Standard
Libraries for that version.
Expand All @@ -53,23 +52,23 @@ def stdlib_list(version=None):
else ".".join(str(x) for x in sys.version_info[:2])
)

module_list_file = os.path.join("lists", "{}.txt".format(version))
module_list_file = os.path.join("lists", f"{version}.txt")

data = pkgutil.get_data("stdlib_list", module_list_file).decode()
data = pkgutil.get_data("stdlib_list", module_list_file).decode() # type: ignore[union-attr]

result = [y for y in [x.strip() for x in data.splitlines()] if y]

return result


@lru_cache(maxsize=16)
def _stdlib_list_with_cache(version=None):
def _stdlib_list_with_cache(version: str | None = None) -> list[str]:
"""Internal cached version of `stdlib_list`"""
return stdlib_list(version=version)


@lru_cache(maxsize=256)
def in_stdlib(module_name, version=None):
def in_stdlib(module_name: str, version: str | None = None) -> bool:
"""
Return a ``bool`` indicating if module ``module_name`` is in the list of stdlib
symbols for python version ``version``. If ``version`` is ``None`` (default), the
Expand Down
Empty file added stdlib_list/py.typed
miketheman marked this conversation as resolved.
Show resolved Hide resolved
Empty file.
1 change: 1 addition & 0 deletions tests/test_base.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import pkgutil

import pytest

import stdlib_list
Expand Down