added support for LlamaLocal's path outside plugin-node/dist #649
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Relates to:
#242
Risks
Low,
User can provide any path for llamalocal and the plugin will direct/load from that directory.
Background
Currently if user tries
pnpm build
, thenode-plugin/dist
gets deleted where the downloaded models are stored.and hence it tries download everytime the build command is executed.
What does this PR do?
This PR will introduce a
LLAMALOCAL_PATH
environment variable, which can be used to point/download llama modelsWhat kind of change is this?
Improvements (misc. changes to existing features)
Documentation changes needed?
My changes require a change to the project documentation.
It will require to add a new environment variable and it's detail.
Testing
Where should a reviewer start?
Detailed testing steps
defaultCharacter.ts
make sure it hasmodelProvider: ModelProviderName.LLAMALOCAL
pnpm build
it will download theNousResearch/Hermes-3-Llama-3.1-8B-GGUF/resolve/main/Hermes-3-Llama-3.1-8B.Q8_0.gguf?download=true
models into directory specified inLLAMALOCAL_PATH
pnpm build
again,Model already exists
Discord username
dexbruce