OLLAMA not using GPU - Jetson Nano 4g

[mahdighaemi123]

Search before asking

• I have searched the jetson-containers issues and found no similar feature requests.

Description

Hi
I have JETSON NANO 4G I'm using Ollama and it is ok with the CPU
I want to use my GPU
how I can do that
I searched for about 10 hours and tested lots of ways but none of them worked
please add a doc for using Ollama with GPU for Jetson Nano 4g thanks
thanks

Use case

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[dusty-nv]

@mahdighaemi123 a couple weeks ago we had updated ollama to be building and working again with CUDA, and had updated container image for JetPack 6.1. Is your request for the JetPack 4.6 version for original Jetson Nano 4GB? It has been awhile building for JetPack 4.6, but it should still build llama.cpp at least. Have you tried llama.cpp? Many of these issues with ollama, they don't appear in llama.cpp. llama.cpp has more direct API and you can clearly enable/disable CUDA, I would try that next.