Allow separation of backend infrastructure from model definitions

[metachip]

Currently gptel combines backend infrastructure (OpenAI API compatibility) with model definitions in a way that makes it difficult for me to fully customize my list model definitions without inheriting the default ones. This is particularly evident when trying to override the default OpenAI models while still using the OpenAI-compatible API infrastructure.

Background:

1. I prefer to define my own complete set of models with custom capabilities, costs, or other parameters.
   • I do this because I want to curate my own list of models available to me, including tips for marginalia.
   • I can also update and modify models at a cadence faster than gptel upstream updates.
   • I do not want models that I will not or cannot use to appear in the list of options.
2. The current architecture makes this difficult because:
   • Default models are defined in gptel--openai which is also the base backend
   • Other backends inherit from this, making it hard to cleanly override
   • Attempts to modify gptel--openai directly could break package functionality

Feature Request:

1. Separate the OpenAI API infrastructure from the model definitions by:

   • Moving the base backend infrastructure to a separate file (as mentioned in Customization of gptel-backend causes recursive require error, prevents gptel from loading #556)
   • Making model definitions purely additive rather than inherited

2. Provide a clean way to define models without inheriting defaults:

   (gptel-define-openai "OpenAI"       ; New suggested name for OpenAI models
     :key 'gptel-api-key
     :stream t
     :models '((my-model-1
                :description "..."
                :capabilities (...))
              (my-model-2
                ...))
     :inherit-models nil)              ; New option to skip default models

3. (Optional) Consider renaming the gptel-make-* functions to gptel-define-* as suggested in Customization of gptel-backend causes recursive require error, prevents gptel from loading #556, since they both create backends and register them in gptel's namespace.

I think this aligns with the ongoing work to reorganize the codebase (moving backend infrastructure to a separate file) while providing a cleaner way for users to fully customize their model definitions.

Benefits:

• Clearer separation of concerns between backend infrastructure and model definitions
• More intuitive way to override default models without breaking functionality
• Better alignment with the package's future direction (v1.0 plans)
• Addresses underlying architectural issues highlighted in Customization of gptel-backend causes recursive require error, prevents gptel from loading #556 and (void-variable gptel--known-backends) when using gptel-make-openai #227

[karthink]

Other backends inherit from this, making it hard to cleanly override

They don't. I think you're misunderstanding how struct inheritance works in elisp.

Provide a clean way to define models without inheriting defaults:

You can define your own OpenAI backend like this:

(gptel-define-openai "my-openai"
  :key 'gptel-api-key
  :stream t
  :models '((my-model-1
             :description "..."
             :capabilities (...))
           (my-model-2
             ...)))

And use it like normal.

I do not want models that I will not or cannot use to appear in the list of options.

If you want to be rid of the default backend, you can remove it from the registry:

(setf (gptel-get-backend "ChatGPT") nil)

Consider renaming the gptel-make-* functions to gptel-define-*

While this is planned for v1.0, it is purely cosmetic and will not change any behaviors. (Its biggest effect will be to annoy users who have to update their configuraitons.)

Clearer separation of concerns between backend infrastructure and model definitions

Backends and models are not tied together in any way. They are currently completely separate.

More intuitive way to override default models without breaking functionality

• You can associate any set of models with any backend.
• You can (put 'gpt-4o-mini :property value) to set whatever metadata you want.

Making model definitions purely additive rather than inherited

Because backends and models are completely separate, model definitions are not inherited.

Each model is an independent object. If you are not happy with its existing metadata you can modify it or add your own, as described above.

Addresses underlying architectural issues

Unfortunately none of the proposed changes will help with the custom-set-variables issue.

Moving the base backend infrastructure to a separate file

Having looked into this, this is not going to fix the problem.

[metachip]

Thanks for all that. I have managed to configured gptel the way I like it now (no ChatGPT default backend and models).

This does work:

(setf (gptel-get-backend "ChatGPT") nil)

when followed by the backend model definitions I want, but strangely only if I locate them outside the (use-package gptel) sexp. If locate them inside this sexp, in the :config clause, it does not work. It seems to be a config/state setup issue in gptel. Locating them as I have (outside and after the use-package sexp) it works but I am interested to know why it won't work inside the :config clause.

This is what I have

(use-package gptel
  ;; b0f78673-2d21-4a1b-b161-e75d3debdff4
  :straight t
  :bind (:map ctrl-z-map ("g" . gptel-send))
  :custom
  (gptel-default-mode 'org-mode)
  (gptel-prompt-prefix-alist nil)
  (gptel-org-branching-context t)
  :config
  (setq gptel-expert-commands t)
  (setf (gptel-get-backend "ChatGPT") nil))

which is followed by:

(gptel-make-gemini "Gemini"
  :protocol "https"
  :host "generativelanguage.googleapis.com"
  :key (lambda () (auth-source-pick-first-password
                    :host "generativelanguage.googleapis.com"
                    :user "gemini-api-key"))
  :stream t
  :models '((gemini-2.0-flash
              :description "Next gen, high speed, multimodal for a diverse variety of tasks"
              :capabilities (tool-use json media)
              :mime-types ("image/png" "image/jpeg" "image/webp" "image/heic" "image/heif"
                            "application/pdf" "text/plain" "text/csv" "text/html")
              :context-window 1000
              :input-cost 0.10
              :output-cost 0.40
              :cutoff-date "2024-08")))
(gptel-make-anthropic "Claude"...)
(gptel-make-openai "Perplexity"...)
(gptel-make-openai "DeepSeek"...)

after the use-package sexp. If instead I do this:

(use-package gptel
  ;; b0f78673-2d21-4a1b-b161-e75d3debdff4
  :straight t
  :bind (:map ctrl-z-map ("g" . gptel-send))
  :custom
  (gptel-default-mode 'org-mode)
  (gptel-prompt-prefix-alist nil)
  (gptel-org-branching-context t)
  :config
  (setq gptel-expert-commands t)
  (setf (gptel-get-backend "ChatGPT") nil)
  (gptel-make-gemini "Gemini"
    :protocol "https"
    :host "generativelanguage.googleapis.com"
    :key (lambda () (auth-source-pick-first-password
                      :host "generativelanguage.googleapis.com"
                      :user "gemini-api-key"))
    :stream t
    :models '((gemini-2.0-flash
                :description "Next gen, high speed, multimodal for a diverse variety of tasks"
                :capabilities (tool-use json media)
                :mime-types ("image/png" "image/jpeg" "image/webp" "image/heic" "image/heif"
                              "application/pdf" "text/plain" "text/csv" "text/html")
                :context-window 1000
                :input-cost 0.10
                :output-cost 0.40
                :cutoff-date "2024-08")))
  (gptel-make-anthropic "Claude"...)
  (gptel-make-openai "Perplexity"...)
  (gptel-make-openai "DeepSeek"...))

It does not work?

[karthink]

Just checking: Are you on the latest gptel commit?

[metachip]

Just checking: Are you on the latest gptel commit?

I tested this at this commit aa649b0.

[karthink]

I made some changes to gptel-get-backend since that commit, so please update and test?

[metachip]

As of

d57eb98 master origin/master gptel-transient: Improve parsing of crowdsourced prompts CSV file (#615)

This hack works

(use-package gptel
  :straight t
  :bind (:map ctrl-z-map ("g" . gptel-send))
  :custom
  (gptel-default-mode 'org-mode)
  (gptel-prompt-prefix-alist nil)
  (gptel-org-branching-context t)
  :config
  (setq gptel-expert-commands t)
  (setf (gptel-get-backend "ChatGPT") nil))
(setq gptel--openai...)
(gptel-make-gemini "Gemini"...)
(gptel-make-anthropic "Claude"...)
(gptel-make-openai "Perplexity"...)
(gptel-make-openai "DeepSeek"...)

but this (preferred) configuration does not:

(use-package gptel
  :straight t
  :bind (:map ctrl-z-map ("g" . gptel-send))
  :custom
  (gptel-default-mode 'org-mode)
  (gptel-prompt-prefix-alist nil)
  (gptel-org-branching-context t)
  :config
  (setq gptel-expert-commands t)
  (setf (gptel-get-backend "ChatGPT") nil)
  (setq gptel--openai...)
  (gptel-make-gemini "Gemini"...)
  (gptel-make-anthropic "Claude"...)
  (gptel-make-openai "Perplexity"...)
  (gptel-make-openai "DeepSeek"...))

That is, if the models are configured in :config inside (use-package gptel...) it does not work but if they are moved outside it and after it, it does.

[karthink]

Sorry, I don't understand why this is happening. I couldn't reproduce it.