Support Python 3.12

[EwoutH]

Python 3.12 releases next week, Monday 2023-10-02. It would be great if vLLM could support Python 3.12 fully, including testing in CI and wheels uploaded to PyPI.

Summary – Python 3.12 Release highlights

New grammar features:

• PEP 701: Syntactic formalization of f-strings

Interpreter improvements:

• PEP 684: A Per-Interpreter GIL
• PEP 669: Low impact monitoring for CPython

New typing features:

• PEP 688: Making the buffer protocol accessible in Python
• PEP 692: Using TypedDict for more precise **kwargs typing
• PEP 695: Type Parameter Syntax
• PEP 698: Override Decorator for Static Typing

Important deprecations, removals or restrictions:

• PEP 623: Remove wstr from Unicode
• PEP 632: Remove the distutils package. See the migration guide for advice on its replacement.
• gh-95299: Do not pre-install setuptools in virtual environments created with venv. This means that distutils, setuptools, pkg_resources, and easy_install will no longer available by default; to access these run pip install setuptools in the activated virtual environment.

[WoosukKwon]

Hi @EwoutH, thanks for bringing this up. Once our dependencies (e.g., PyTorch, Ray) support python 3.12, we will support it as well.

[kannon92]

Ray still seems to be not supporting python 3.12 yet: ray-project/ray#40211

[greg2451]

Two days ago, PR to support 3.12 for ray was merged: ray-project/ray#45621. We can hope support will arrive soon enough.

But as mentioned:

TL;DR; this PR will support python 3.12 as a nightly wheel; other release artifacts will take more work

[beniz]

Modifying CMakeLists.txt for Python 3.12 works fine:

diff --git a/CMakeLists.txt b/CMakeLists.txt
index aa15b632..a930f13e 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -13,7 +13,7 @@ include(${CMAKE_CURRENT_LIST_DIR}/cmake/utils.cmake)
 # Supported python versions.  These versions will be searched in order, the
 # first match will be selected.  These should be kept in sync with setup.py.
 #
-set(PYTHON_SUPPORTED_VERSIONS "3.8" "3.9" "3.10" "3.11")
+set(PYTHON_SUPPORTED_VERSIONS "3.8" "3.9" "3.10" "3.11" "3.12")

 # Supported NVIDIA architectures.
 set(CUDA_SUPPORTED_ARCHS "7.0;7.5;8.0;8.6;8.9;9.0")

[rockwotj]

I was able to build for python 3.12 as well