[Feature] Extend GPT4All API to Interact with Local Ollama Server

[sankalpsp98]

Currently, the GPT4All API server allows interaction with models loaded within the GPT4All application . However, it would be beneficial to extend the API to allow interaction with a local Ollama server. This would enable users to leverage the strengths of both platforms:  

GPT4All: User-friendly interface, LocalDocs integration, and existing API infrastructure.
Ollama: Efficient performance, diverse model support (Llama, Mistral, Gemma, Phi), and streamlined model management with Modelfiles .  
This integration would provide users with greater flexibility in choosing and managing their preferred LLMs, combining the ease of use of GPT4All with the performance and model diversity of Ollama.

Proposed Implementation:

One possible approach is to allow users to specify an Ollama server endpoint in the GPT4All API settings. GPT4All could then proxy API requests to the specified Ollama server, enabling seamless interaction with Ollama models. This could be achieved by:

Adding a new setting in the GPT4All application to specify the Ollama server address (e.g., http://localhost:11434).
Modifying the GPT4All API server to forward requests to the configured Ollama server when an Ollama model is specified.
Ensuring compatibility between the GPT4All API and the Ollama API, potentially through an adapter layer to handle any differences in request/response formats.
Benefits:

### Expanded Model Support: Access a wider range of LLMs, including those optimized for different tasks and hardware.
Improved Performance: Potentially leverage Ollama's performance optimizations for faster inference.
Simplified Model Management: Utilize Ollama's Modelfile format for easier model downloading and management.
Increased Flexibility: Choose the best platform for specific needs and preferences.
This enhancement would significantly broaden the capabilities of the GPT4All API and provide a more versatile and powerful solution for users running LLMs locally.

[benja0x40]

This feature request suggests to develop GPT4All in a similar direction as AnythingLLM which can connect to various LLM servers, either remote or local (e.g. Ollama or LMStudio), in addition to providing a builtin basic LLM inference engine.

Regarding local inference engines, LMStudio has recently taken the lead thanks to multiple backend support and modular runtime updates. To my knowledge this is the best available solution, maximising the range of supported models and hardwares while keeping up with the speed at which the field is progressing (llama.cpp has multiple releases per week).

I don't know which approach to recommend as these are strategic choices which depend on GPT4All's development goals in the long run.