[Doc] Update description of vLLM support for CPUs

[DamonFool]

Hi all,

The description of CPU support in vLLM is out of date.
It would be better to update it.

Thanks.
Best regards,
Jie

[mgoin]

Since the AVX2 and PowerPC backends don't have active testing and this documentation assumes you are building on a machine with AVX512, I'm a little hesitant to update in this doc. Maybe if you call out specifically where you can get directions for AVX2 and PowerPC, and that this doc is assuming AVX512, that would be more clear.

[DamonFool]

Thanks @mgoin for the review.

The origin description AVX512 is required is quite misleading, which means vLLM requires AVX512 ISA to run.
It would limit the usage of vLLM if people wrongly get the msg that vLLM only supports AVX512.

Actually, we've built and run vLLM on AVX2-only machines.
So I would suggest removing AVX512 is required here.

However, if you're against adding the PowerPC CPUs here, I'm fine.
This is because we don't have PowerPC CPUs and don't test it at all.

Thanks.

[DamonFool]

Since the AVX2 and PowerPC backends don't have active testing and this documentation assumes you are building on a machine with AVX512, I'm a little hesitant to update in this doc. Maybe if you call out specifically where you can get directions for AVX2 and PowerPC, and that this doc is assuming AVX512, that would be more clear.

Hi @mgoin , I kept AVX512 in the doc.
But make it to be (optional, recommended) as the description of the compiler.
What do you think?
Thanks.

[DamonFool]

Hi @mgoin , are you fine with the current change in cpu-installation.rst?
Thanks.

[mgoin]

Hi @DamonFool, yes this is fine! It would be nice to make a new section for avx2, but I'll leave that up to you.

[DamonFool]

Hi @DamonFool, yes this is fine! It would be nice to make a new section for avx2, but I'll leave that up to you.

The patch aims to tell people that AVX512 is not required for vLLM on x86 CPU (and you can also try it on avx2 too).

There seems no difference to build and run vLLM on avx512 and avx2.
So I'm not sure what should I add for the avx2 section.
Any suggestions?
Thanks.

[DamonFool]

Hi @WoosukKwon , are you OK with this change?
Thanks.

[DamonFool]

Hi @mgoin , @WoosukKwon seems to be busy with other things.
Do you think it's fine to get this doc-only change merged regarding that there is no objection from the community?
Thanks.

[mgoin]

@DamonFool Yes, I am simply waiting for your check to be green. I cannot merge without passing checks. Please try merging with main

[DamonFool]

Please try merging with main

Done.
Thanks.

[DamonFool]

All the required CI tests had been passed. @mgoin
Thanks.

[WoosukKwon]

I'm good with this doc change, but a little bit worried about the potential confusion and complexity as the Intel team will be adding IPEX or other intel-cpu-only optimizations to the cpu backend.

[DamonFool]

I'm good with this doc change, but a little bit worried about the potential confusion and complexity as the Intel team will be adding IPEX or other intel-cpu-only optimizations to the cpu backend.

Thanks @WoosukKwon .

You mean vllm-cpu on intel may run faster than other cpus?
Since vLLM can already run on AMD and PowerPC cpus, it would be good to let people know that fact.
Then, vLLM would be used more widely.
And we can do more opts on other cpus when necessary in the future, right?

[WoosukKwon]

@DamonFool Sorry for misleading you. Yes this PR doesn't have a problem. I just wanted to say we'll need to figure out how to efficiently maintain the Intel and non-Intel CPU backends while not complicating the code.

[DamonFool]

@DamonFool Sorry for misleading you. Yes this PR doesn't have a problem. I just wanted to say we'll need to figure out how to efficiently maintain the Intel and non-Intel CPU backends while not complicating the code.

Got it.
Thank you all for the help.