Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BFCL] Bug Fix parse_nested_value function for model_handler utils #660

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions berkeley-function-call-leaderboard/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

All notable changes to the Berkeley Function Calling Leaderboard will be documented in this file.

- [Sept 25, 2024] [#660](https://github.com/ShishirPatil/gorilla/pull/660): Bug fix in `parse_nested_value` function to handle nested dictionary values properly.
- [Sept 19, 2024] [#644](https://github.com/ShishirPatil/gorilla/pull/644): BFCL V3 release:
- Introduce new multi-turn dataset and state-based evaluation metric
- Separate ast_checker and executable_checker for readability
Expand Down
19 changes: 13 additions & 6 deletions berkeley-function-call-leaderboard/bfcl/model_handler/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -751,19 +751,25 @@ def parse_nested_value(value):
Parse a potentially nested value from the AST output.

Args:
value: The value to parse, which could be a nested dictionary or a simple value.
value: The value to parse, which could be a nested dictionary, which includes another function call, or a simple value.

Returns:
str: A string representation of the value, handling nested function calls.
str: A string representation of the value, handling nested function calls and nested dictionary function arguments.
"""
if isinstance(value, dict):
func_name = list(value.keys())[0]
args = value[func_name]
args_str = ", ".join(f"{k}={parse_nested_value(v)}" for k, v in args.items())
return f"{func_name}({args_str})"
# Check if the dictionary represents a function call (i.e., the value is another dictionary or complex structure)
if all(isinstance(v, dict) for v in value.values()):
func_name = list(value.keys())[0]
args = value[func_name]
args_str = ", ".join(f"{k}={parse_nested_value(v)}" for k, v in args.items())
return f"{func_name}({args_str})"
else:
# If it's a simple dictionary, treat it as key-value pairs
return "{" + ", ".join(f"'{k}': {parse_nested_value(v)}" for k, v in value.items()) + "}"
return repr(value)



def decoded_output_to_execution_list(decoded_output):
"""
Convert decoded output to a list of executable function calls.
Expand All @@ -782,3 +788,4 @@ def decoded_output_to_execution_list(decoded_output):
)
execution_list.append(f"{key}({args_str})")
return execution_list