Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not respecting OpenAI model selection #1105

Closed
Kevin-Mok opened this issue Dec 14, 2024 · 2 comments
Closed

Not respecting OpenAI model selection #1105

Kevin-Mok opened this issue Dec 14, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@Kevin-Mok
Copy link

Kevin-Mok commented Dec 14, 2024

Describe the bug

The bot is responding twice, one with the correct model (small) and another with an unselected model (large) despite setting it the character file as such from #845, reverting the hardcoded value from #853 (Kevin-Mok@466782c) and setting [SMALL|MEDIUM|LARGE]_OPENAI_MODEL=gpt-4o-mini in .env.

To Reproduce

  1. git clone https://github.com/Kevin-Mok/eliza.git
    
  2. Set .env like so.
    SMALL_OPENAI_MODEL=gpt-4o-mini
    MEDIUM_OPENAI_MODEL=gpt-4o-mini
    LARGE_OPENAI_MODEL=gpt-4o-mini
    
  3. Set the character config like so.
    {
    "modelProvider": "openai",
      "settings": {
        "model": "small"
      }
    }
    
  4. pnpm start --character="path/to/character.json"

Expected behavior

Only the small model should be used.

Logs

Pre-Large Usage

 ◎ LOGS
   Creating Memory 
   2584cfba-64a2-0306-8b33-0b5958bcc298 
   youre stuck 

Message: {
  content: {
    text: 'youre stuck',
    attachments: [],
    source: 'discord',
    url: 'https://discord.com/channels/401422081025245184/1315441204715851826/1317575064295833640',
    inReplyTo: '96bb78ce-e84e-031f-9946-c0566ef94bb4'
  },
  userId: '6b35833a-b50c-0d49-8449-0941f15cf0ee',
  agentId: '03419646-efb1-0b57-a0ec-0027c41d97ae',
  roomId: 'a3464273-b491-037e-992c-a1e51f03ed03'
}
Validating transfer from user: 6b35833a-b50c-0d49-8449-0941f15cf0ee
Message: {
  content: {
    text: 'youre stuck',
    attachments: [],
    source: 'discord',
    url: 'https://discord.com/channels/401422081025245184/1315441204715851826/1317575064295833640',
    inReplyTo: '96bb78ce-e84e-031f-9946-c0566ef94bb4'
  },
  userId: '6b35833a-b50c-0d49-8449-0941f15cf0ee',
  agentId: '03419646-efb1-0b57-a0ec-0027c41d97ae',
  roomId: 'a3464273-b491-037e-992c-a1e51f03ed03'
}

 ["◎ Generating message response.."] 

 ["◎ Generating text..."] 

 ℹ INFORMATIONS
   Generating text with options: 
   {"modelProvider":"openai","model":"small"} 

 ℹ INFORMATIONS
   Selected model: 
   gpt-4o-mini 

 ◎ LOGS
   Creating Memory 
   3ae62ff6-70a5-0972-9d38-8cb2ed3aedcc 
   stuck? nah, just recharging my creative circuits! what’s got you feeling that way? 

 ["✓ Normalized action: continue"] 

 ["ℹ Executing handler for action: CONTINUE"] 

 ["◎ Generating text..."] 

 ℹ INFORMATIONS
   Generating text with options: 
   {"modelProvider":"openai","model":"small"} 

 ℹ INFORMATIONS
   Selected model: 
   gpt-4o-mini 

 ["◎ Generating message response.."] 

Large Model Usage (cont. from above)

 ["◎ Generating text..."] 

 ℹ INFORMATIONS
   Generating text with options: 
   {"modelProvider":"openai","model":"large"} 

 ℹ INFORMATIONS
   Selected model: 
   gpt-4o 

 ◎ LOGS
   Creating Memory 
   975c6fda-78f3-0bf0-b6f9-ee7780e86797 
   nah, just recharging my creative circuits! what's got you feeling that way? maybe we can troubleshoot it together. 

 ◎ LOGS
   Evaluating 
   GET_FACTS 

 ◎ LOGS
   Evaluating 
   UPDATE_GOAL 

 ◎ LOGS
   Evaluating 
   EXTRACT_RECOMMENDATIONS 

 ["◎ Generating text..."] 

 ℹ INFORMATIONS
   Generating text with options: 
   {"modelProvider":"openai","model":"small"} 

 ℹ INFORMATIONS
   Selected model: 
   gpt-4o-mini 

 ["◎ Generating text..."] 

 ℹ INFORMATIONS
   Generating text with options: 
   {"modelProvider":"openai","model":"small"} 

 ℹ INFORMATIONS
   Selected model: 
   gpt-4o-mini 

Skipping process
@Kevin-Mok Kevin-Mok added the bug Something isn't working label Dec 14, 2024
@Kevin-Mok Kevin-Mok changed the title Not respecting model selection Not respecting OpenAI model selection Dec 14, 2024
@oxSaturn
Copy link
Contributor

oxSaturn commented Dec 15, 2024

Try to sync your main branch to the latest commit and try again, I belive configuring in .env should work fine. See PR here #999.

Screenshot-4uErmvrt@2x

@Kevin-Mok
Copy link
Author

This worked, thanks. I'm assuming it respects my LARGE_OPENAI_MODEL=gpt-4o-mini in .env due to #999 now. Thanks for the PR!

 ["◎ Generating text..."] 

 ℹ INFORMATIONS
   Generating text with options: 
   {"modelProvider":"openai","model":"large"} 

 ℹ INFORMATIONS
   Selected model: 
   gpt-4o-mini

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants