Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: use sgl-kernel by default #3088

Closed
wants to merge 6 commits into from
Closed

feat: use sgl-kernel by default #3088

wants to merge 6 commits into from

Conversation

zhyncs
Copy link
Member

@zhyncs zhyncs commented Jan 23, 2025

Motivation

cc @merrymercy @Ying1123

Modifications

Checklist

Sorry, something went wrong.

@zhyncs
Copy link
Member Author

zhyncs commented Jan 23, 2025

FYI We plan to remove this configuration and the else branch in the next major release.

@zhyncs
Copy link
Member Author

zhyncs commented Jan 24, 2025

FYI There are precision issues with fused_rms_norm in bf16 in both FlashInfer 0.1.6 and the latest main. I will fix this issue in sgl-kernel and then merge this PR.

@zhyncs zhyncs marked this pull request as draft January 25, 2025 16:25
@zhyncs zhyncs closed this Jan 26, 2025
@zhyncs zhyncs deleted the zhyncs/sgl branch January 26, 2025 11:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant