Is there anyway to route the plugin to a privateGPT instance hosted locally on the same computer or on a local network?

[dziliak]

I love the idea of using this plugin on an offline LLM instead of giving my data to the cloud. Are there any suggestions on where to look in this code and other resources to kludge something together to use privateGPT instead of openAI?

I didn't see anyone else asking this and hope this is the right spot to ask it.

[BradKML]

Seconded but for something like Oobabooga (or Ollama once they get on windows)... or stronger integration with GPT4All if their limits can be tolerated. Not sure about LocalChat and Local AI
Same applies to debanjandhar12/logseq-chatgpt-plugin#32

[Green-Sky]

llama.cpp's server provides a more or less openai compatible api, so making the api url configurable might be all thats needed.

[briansunter]

This should already be compatible with an openai compatible endpoint, even one locally. I do have ollama but haven't gotten around to testing it yet, but it should just be updating the settings to point at a local api endpoint. If someone wanted to test and let me know if there are any limitations or anything else needed, that would be great!

[traverseda]

Can confirm this works with

  "openAIKey": "",
  "openAICompletionEngine": "llama3.2",
  "chatCompletionEndpoint": "http://localhost:11434/v1",

Don't forget to pull the model you want to use separately. ollama pull llama3.2.

Haven't done any extensive testing because the only thing I'd want a plugin like this is for generating summaries, and #75 is needed for that.