Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ollama GPU support #299

Merged
merged 1 commit into from
Nov 30, 2024
Merged

Ollama GPU support #299

merged 1 commit into from
Nov 30, 2024

Conversation

aaronpowell
Copy link
Member

Added a new method WithGPUSupport to the extension methods for when you create an Ollama server resource. Using an enum for the GPU support options.

Closes #298

PR Checklist

  • Created a feature/dev branch in your fork (vs. submitting directly from a commit on main)
  • Based off latest main branch of toolkit
  • PR doesn't include merge commits (always rebase on top of our main, if needed)
  • New integration
    • Docs are written
    • Added description of major feature to project description for NuGet package (4000 total character limit, so don't push entire description over that)
  • Tests for the changes have been added (for bug fixes / features) (if applicable)
  • Contains NO breaking changes
  • Every new API (including internal ones) has full XML docs
  • Code follows all style conventions

Other information

@aaronpowell
Implementing a method to enable GPU support on ollama
2926446 
Using an enum for the GPU choice, defaulting to nvidia though.

Fixes #298
@aaronpowell
Copy link
Member Author

@elbruno here's the GPU support that we discussed.

@github-actions GitHub Actions
Copy link

Code Coverage

Package Line Rate Branch Rate Complexity Health
CommunityToolkit.Aspire.Hosting.ActiveMQ 60% 24% 104 ❌
CommunityToolkit.Aspire.Hosting.ActiveMQ.MassTransit 1% 0% 14 ❌
CommunityToolkit.Aspire.Hosting.Azure.DataApiBuilder 100% 100% 22 ✔
CommunityToolkit.Aspire.Hosting.Azure.StaticWebApps 100% 100% 28 ✔
CommunityToolkit.Aspire.Hosting.Bun 81% 71% 54 ✔
CommunityToolkit.Aspire.Hosting.Deno 84% 75% 72 ✔
CommunityToolkit.Aspire.Hosting.Golang 94% 50% 16 ✔
CommunityToolkit.Aspire.Hosting.Java 98% 71% 58 ✔
CommunityToolkit.Aspire.Hosting.Meilisearch 61% 27% 94 ➖
CommunityToolkit.Aspire.Hosting.NodeJS.Extensions 90% 68% 92 ✔
CommunityToolkit.Aspire.Hosting.Ollama 65% 64% 198 ➖
CommunityToolkit.Aspire.Hosting.Python.Extensions 67% 46% 56 ➖
CommunityToolkit.Aspire.Hosting.Rust 94% 83% 16 ✔
CommunityToolkit.Aspire.Hosting.SqlDatabaseProjects 72% 64% 58 ➖
CommunityToolkit.Aspire.Meilisearch 97% 92% 68 ✔
CommunityToolkit.Aspire.OllamaSharp 84% 77% 76 ✔
Summary 74% (2050 / 2770) 60% (484 / 808) 1026 ➖

Minimum allowed line rate is 60%

@elbruno
Copy link

elbruno commented Nov 29, 2024

@elbruno here's the GPU support that we discussed.

Amazing! thanks

@aaronpowell aaronpowell merged commit c11ae02 into main Nov 30, 2024
10 checks passed
@aaronpowell aaronpowell deleted the aaronpowell/issue298 branch November 30, 2024 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

API to enable GPU support in Ollama
2 participants
@aaronpowell @elbruno