New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Better handling for configurations that WebLLM doesn't work on. #4

Open

erikh2000 opened this issue Jan 1, 2025 · 0 comments

Owner

erikh2000 commented Jan 1, 2025

There are some devices where WebLLM starts loading and never finishes, sometimes due to lack of video ram.

Example - my Fairphone will load the model to persistent storage, and then stop at 4gb loading it into GPU.

Can I do something to detect this in advance and stop / advise against the load?

Can I recover from a failed load?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment