[Usage]:

[h7878778h]

Your current environment

The output of above commands

How would you like to use vllm on ascend

I refer to this guide: https://vllm-ascend.readthedocs.io/en/latest/quick_start.html

I have pulled Image: quay.io/ascend/vllm-ascend:v0.7.1rc1
run this image on Ascend environment:

run command: vllm serve {model_path}
got an error:

   ...
   raise RuntimeError("Failed to infer device type")
   RuntimeError: Failed to infer device type

I look at vllm source code, it seems that NPU is not supported?

How can I fix this problem？

[wangxiyuan]

can you try the command in python to see if there is any error?

import torch
import torch_npu

[h7878778h]

got another error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/python3.10/lib/python3.10/site-packages/torch_npu/__init__.py", line 19, in <module>
    from torch_npu.utils._error_code import ErrCode, pta_error
  File "/usr/local/python3.10/lib/python3.10/site-packages/torch_npu/utils/__init__.py", line 1, in <module>
    from torch_npu import _C
ImportError: libhccl.so: cannot open shared object file: No such file or directory