[Bug]: How to fix this error No module named 'llama_index.llms.llama_cpp'

[deecancode]

Bug Description

Hi all,

I am trying to use mixtral-8x7b with my own data with no luck.

Version

0.10.25

Steps to Reproduce

import torch
from llama_index.llms.llama_cpp import LlamaCPP
from llama_index.llms.llama_cpp.llama_utils import messages_to_prompt, completion_to_prompt
llm = LlamaCPP(
    model_url=None, # We'll load locally.
    model_path='./Models/mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf', # 4-bit model
    temperature=0.1,
    max_new_tokens=1024, # Increasing to support longer responses
    context_window=8192, # Mistral7B has an 8K context-window
    generate_kwargs={},
    # set to at least 1 to use GPU
    model_kwargs={"n_gpu_layers": 40}, # 40 was a good amount of layers for the RTX 3090, you may need to decrease yours if you have less VRAM than 24GB
    messages_to_prompt=messages_to_prompt,
    completion_to_prompt=completion_to_prompt,
    verbose=True
)

Relevant Logs/Tracbacks

Which gives error No module named 'llama_index.llms.llama_cpp'.

I have installed llama_index, used my MAC Mini and also Google Colab's GPUs

Any suggestions?

[dosubot]

Hey @deecancode, great to see you diving back into the world of LlamaIndex! Running into a bit of trouble, I see? Let's see what we can do about that.

Based on the error message you provided, it seems like the Python interpreter is unable to find the llama_cpp module within the llama_index.llms package. This could be due to several reasons, such as the module not being installed, being installed in a different Python environment, or a typo in the module name.

However, after reviewing the context provided, it appears that the issue is related to the import statement from llama_index.llms.llama_cpp import LlamaCPP. The error message you're encountering is likely a misinterpretation or a simplification of the actual ImportError raised by this line.

To resolve this issue, you should ensure that the llama-cpp-python package is correctly installed in your Python environment. The error message within the except ImportError block provides a helpful hint on how to resolve this:

raise ImportError(
    "Could not import llama_cpp library."
    "Please install llama_cpp with `pip install llama-cpp-python`."
    "See the full installation guide for GPU support at "
    "`https://github.com/abetlen/llama-cpp-python`"
)

Therefore, the steps to resolve the issue are as follows:

1. Ensure the correct Python environment: Make sure you're working within the correct Python environment where you intend to use the LlamaIndex library. This is especially important if you're working with virtual environments or if there are multiple Python installations on your system.

2. Install the llama-cpp-python package: Run the following command to install the llama-cpp-python package. This command should be executed in the terminal (for macOS/Linux) or command prompt/PowerShell (for Windows), ensuring that it's run within the context of the correct Python environment.

   pip install llama-cpp-python
   

3. Verify the installation: After installation, you can verify that the llama_cpp module is correctly installed by attempting to import it in a Python shell or script:

   try:
       from llama_cpp import Llama
       print("llama_cpp successfully imported!")
   except ImportError:
       print("Failed to import llama_cpp.")

If you follow these steps and ensure that the llama-cpp-python package is correctly installed in your Python environment, the "No module named 'llama_index.llms.llama_cpp'" error should be resolved.

Regarding your question about the changes made in the LlamaIndex library from version 0.10.25 to 0.10.26 that could potentially affect the import of the 'llama_index.llms.llama_cpp' module, I wasn't able to find any specific changes within the repository that could have caused this issue. However, it's possible that there were changes made elsewhere that I missed. If the issue persists after following the steps above, please provide more information so I can further assist you.

For more details, you can refer to the llama-index-legacy/llama_index/legacy/llms/llama_cpp.py file in the LlamaIndex repository.

Sources

• [Bug]: No module named 'llama_index.core.llms.llama_utils'
• llama-index-legacy/llama_index/legacy/llms/llama_cpp.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[logan-markewich]

pip install llama-index-llms-llama-cpp -- every integration is a package

Tbh though, I would use Ollama, the setup is 1000x easier (similar install, pip install llama-index-llms-ollama)

[sykuann]

i have the same issue and tried CMAKE_ARGS="-DLLAMA_METAL=on" FORCE_CMAKE=1 pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir but still it doesn't work.

[logan-markewich]

@sykuann that only installs llama cpp itself, you still need to install the integration (which I've shown above)