Add support for thinking LLMs directly in gr.ChatInterface
#10305
Conversation
* group thoughts when nested
| @@ -37,6 +37,8 @@ class MetadataDict(TypedDict): | |||
| title: Union[str, None] | |||
| id: NotRequired[int | str] | |||
| parent_id: NotRequired[int | str] | |||
| duration: NotRequired[int] | |||
| status: NotRequired[Literal["pending", "done"]] | |||
There was a problem hiding this comment.
Note that I moved duration inside of metadata where I think it makes more sense. I also added a status for more control over the loading spinner, as it improves the UI a bit (cc @yvrjsharma)
|
Will fix the tests tomorrow but should otherwise be ready for review |
There was a problem hiding this comment.
Docs changes look good, but should we add docs for the gr.ChatMessage data class? Similar to how we did for Audio and ImageEditor?
| @@ -87,14 +88,15 @@ class Metadata(GradioModel): | |||
| title: Optional[str] = None | |||
| id: Optional[int | str] = None | |||
| parent_id: Optional[int | str] = None | |||
| duration: Optional[float] = None | |||
There was a problem hiding this comment.
should it be float or int? it's an int on line 40
There was a problem hiding this comment.
it can be a float in the general case, I'll update the other reference!
yeah good point, I'll expose this! |
|
@abidlabs I added ChatMessage to the docs page for Chatbot as a helper class. The docs are mostly a placeholder though so please correct/rewrite them. |
|
Sweet thanks everyone for the reviews and contribution! Everything should be addressed now I'll merge in once CI runs |
This PR builds off of #10226 to add support for thoughts directly in
gr.ChatInterface. I actually spent a lot of time trying to get the duration to show up automatically based on timing the chat function calls but in the end, everything I tried seemed very "magical" and not too configurable and so I left it to the dev to manually set it.To test, try these two demos:
demo/chatinterface_thoughts/run.pydemo/chatinterface_nested_thoughts/run.pyThis PR also essentially closes: #9051, or at least the most actionable part of it -- increased documentation on how to use thinking LLMs with
gr.ChatInterface