Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support FP32 #72

Closed
WoosukKwon opened this issue May 5, 2023 · 0 comments · Fixed by #141
Closed

Support FP32 #72

WoosukKwon opened this issue May 5, 2023 · 0 comments · Fixed by #141

Comments

@WoosukKwon
Copy link
Collaborator 

          Yes, it does. It is our attention kernel that does not support FP32. More precisely, our attention kernel currently does not support some block sizes when FP32 is used. I will fix this in the future.

Originally posted by @WoosukKwon in #70 (comment)
@WoosukKwon WoosukKwon added new model Requests to new models and removed new model Requests to new models labels May 11, 2023
@WoosukKwon WoosukKwon mentioned this issue Jun 6, 2023
Merged
yukavio pushed a commit to yukavio/vllm that referenced this issue Jul 3, 2024
@jeanniefinks
Jfinks license (vllm-project#72)
197f854 
SUMMARY:
* Renamed forked LICENSE to LICENSE-APACHE
* Updated LICENSE to include new Neural Magic Community License ** will
need help in having metadata not say Apache at top of GitHub window on
main branch as original was forked and it's inheriting this invisible
header on jfinks-license branch
* Created NOTICE to be consistent with Neural Magic repo content

TEST PLAN:
Content reviewed offline with Brian and Rob Shaw; however, need to
resolve final LICENSE file display so it does not indicate Apache
license type
dllehr-amd pushed a commit to dllehr-amd/vllm that referenced this issue Jul 22, 2024
@mawong-amd
Revert "Merge branch 'main' of github.com:ROCm/vllm" (vllm-project#72)
596d58c 
This reverts commit 2a3cbf9, reversing
changes made to 367aa5a.
JHLEE17 pushed a commit to JHLEE17/vllm that referenced this issue Aug 1, 2024
@jkaniecki
Update ops.py (vllm-project#72)
2728599
pi314ever pushed a commit to pi314ever/vllm that referenced this issue Jan 17, 2025
@mfylcek @michalkuligowski
Set vllm-hpu-extension to 6ac93fb (vllm-project#684)
885c60d 
remove expert_max hard code (vllm-project#47)
vLLM-Ext: Full enabling of ALiBi (vllm-project#34)
Add version inference via setuptools-scm (vllm-project#58)
Revert "vLLM-Ext: Full enabling of ALiBi (vllm-project#34)" (vllm-project#59)
Remove punica_hpu.py from vllm_hpu_extension (vllm-project#66)
Removed previous (not-pipelined) pa implementation (vllm-project#72)
Add flag to enable running softmax in fp32 (vllm-project#71)
Update calibration readme link (vllm-project#73)
allow lm_head quantization in calibration process (vllm-project#65)
Pad to bmin if value is less (vllm-project#67)
Update pyproject.toml (HabanaAI#75)

---------

Co-authored-by: Michał Kuligowski <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Support FP32 vllm-project/vllm
1 participant
@WoosukKwon