Python Version Compatibility with Latest Text Generation WebUI Main Branch -- Cuda 12 env.

[strikeoncmputrz]

Hey,

This project looks awesome. I'd really like to use it and contribute.

I've been trying to get it to interoperate with text gen webui commit 3fd707380868ad6a2ba57aaed4c96799c2441d99, which is currently the main branch, and Cuda 12.1.1.

I was curious about the faiss-cpu Python < 3.11 dependency.

If I install a fresh clone of Text Generation Web UI via start_linux.sh and select NVIDIA, the default Conda environment installs Python 3.11.5. If I then clone the LLM_Web_search repo into the extensions directory and follow the install procedures in the readme, the Conda install succeeds, but it breaks text generation web ui dependencies.

me@llama:~/text-generation-webui$
 ./start_linux.sh                                                                             
╭──────────────────────────── Traceback (most recent call last) ─────────────────────────────╮
│ /home/me/text-generation-webui/server.py:6 in <module>                                 │
│                                                                                            │
│     5                                                                                      │
│ ❱   6 import accelerate  # This early import makes Intel GPUs happy                        │
│     7                                                                                      │
╰────────────────────────────────────────────────────────────────────────────────────────────╯
ModuleNotFoundError: No module named 'accelerate'                                             

I assume this is due to downgrading to Python 3.10.x since all the text generation webui dependencies were installed in site packages for Python 3.11.5, but I'm not certain. It looks like text generation webui manual procedures also include installing Python 3.11.5.

I'm curious if other folks are able to get this extension working with a recent commit of text generation webui configured with the cuda back end. If the workaround is forcing text generation webui to install dependencies for Python 3.10.x I'd be happy to try. Hoping this is a me problem :-)

I reverted the conda revision to prior to installing LLM_Web_search dependencies and was able to install failss-cpu==1.7.4 via the conda-forge channel without downgrading to Python 3.10.

(/home/me/text-generation-webui/installer_files/env) me@llama:~/text-generation-webui/extensions/LLM_Web_search$ python -V
Python 3.11.5
(/home/me/text-generation-webui/installer_files/env) me@llama:~/text-generation-webui/extensions/LLM_Web_search$ python -c 'import faiss'
(/home/me/text-generation-webui/installer_files/env) me@llama:~/text-generation-webui/extensions/LLM_Web_search$ 

[mamei16]

Hi, I'm glad to hear to hear you like the project! Thanks for creating this issue and already looking into it.

It appears that the unofficial faiss-wheels, just a week before I started this project, created a release supporting python 3.11. However, the official installation instructions still explicitly mention that conda packages are built only for python 3.8-3.10, so I assumed that there must be a good reason for excluding newer python versions. However, it appears that this restriction is only relevant for the GPU version, meaning I can safely remove the Python < 3.11 dependency.

Unfortunately, I was able to replicate your problem exactly, so thanks a lot for reaching out and pointing out that it works fine with python 3.11.
I pushed commit dbd929d and did a complete re-install of the webui using the newest commit and the extension. It seems to work as intended now! 🎉

[strikeoncmputrz]

Outstanding! I was able to install the extension no problem after pulling your latest commit!

[mamei16]

Awesome, thanks again!