A configuration for deepinfra.com #478

rikhuijzer started this conversation in Show and tell

rikhuijzer
Apr 30, 2024

The following configuration can be used to get Llama 3 70B running on deepinfra.com.

Add the following to extra-openai-models.yaml:

- model_id: 'deepinfra-llama3'
  model_name: 'meta-llama/Meta-Llama-3-70B-Instruct'
  api_base: 'https://api.deepinfra.com/v1/openai'
  api_key_name: 'deepinfra'
  completion: false

and set the key via

$ llm keys set deepinfra

Replies: 0 comments

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment