Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Automated] Merge release into main #214

Merged
merged 31 commits into from
Oct 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
c7603eb
bump the version, test release to PyPi
ProKil May 28, 2024
ea4c128
Update README.md
ProKil May 28, 2024
57beef5
Update README.md
ProKil May 28, 2024
a2265f6
Update README.md
ProKil May 28, 2024
13a7721
Merge remote-tracking branch 'origin/main' into release
ProKil Jun 2, 2024
e798e8c
Merge remote-tracking branch 'origin/main' into release
ProKil Jun 14, 2024
db9ace3
bumpy version to 0.0.9
ProKil Jun 14, 2024
4d4ed5c
Update Sotopia presentation information in README.md
ProKil Jun 17, 2024
859405a
Merge branch 'main' into release
ProKil Jun 18, 2024
fa1a410
bump version to 0.0.10
ProKil Jun 18, 2024
d34115b
Merge remote-tracking branch 'origin/main' into release
ProKil Jun 27, 2024
ff58645
bump version
ProKil Jun 27, 2024
37ccb7b
add merge release back to main action
ProKil Jul 20, 2024
1c05438
change checkout v4->v3
ProKil Jul 20, 2024
e4f0a26
fix merge-back-to-main and pin mypy to <1.11.0
ProKil Jul 20, 2024
865e8b6
Merge branch 'main' into release
XuhuiZhou Aug 26, 2024
7bb0bce
Merge remote-tracking branch 'origin/main' into release
ProKil Sep 2, 2024
88d043d
merge bug fix
ProKil Sep 2, 2024
c9c411c
upgrade default model to handle bad-foratted outputs to gpt-4o-mini a…
yangalan123 Sep 5, 2024
fbe8410
update pull request -> pull request target
ProKil Sep 5, 2024
e3e5737
Merge branch 'release' of github.com:sotopia-lab/sotopia into release
ProKil Sep 5, 2024
78e8eb8
bump version
ProKil Sep 5, 2024
b8a6dbc
Add `bad_output_process_model` option and `use_fixed_model_version` o…
yangalan123 Sep 24, 2024
feef903
fix gpt-3.5
XuhuiZhou Sep 27, 2024
dc6db94
Merge branch 'main' into release
XuhuiZhou Sep 27, 2024
a73ff48
replace gpt3.5 turbo for tests
XuhuiZhou Sep 27, 2024
1923c18
update gpt-3.5-turbo to gpt-4o-mini
ProKil Oct 1, 2024
db8839e
bug fix for return fixed model version function
ProKil Oct 1, 2024
0fcfcb0
fix sampling error
XuhuiZhou Oct 4, 2024
cd31c72
fix rc.4
XuhuiZhou Oct 4, 2024
12e74a6
new tag
XuhuiZhou Oct 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@ asyncio.run(
run_async_server(
model_dict={
"env": "gpt-4",
"agent1": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"agent1": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
},
sampler=UniformSampler(),
)
Expand Down
4 changes: 2 additions & 2 deletions docs/pages/concepts/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ class LLMAgent(BaseAgent[Observation, AgentAction]):
agent_name: str | None = None,
uuid_str: str | None = None,
agent_profile: AgentProfile | None = None,
model_name: str = "gpt-3.5-turbo",
model_name: str = "gpt-4o-mini",
script_like: bool = False,
) -> None:
```
Expand All @@ -26,7 +26,7 @@ class ScriptWritingAgent(LLMAgent):
agent_name: str | None = None,
uuid_str: str | None = None,
agent_profile: AgentProfile | None = None,
model_name: str = "gpt-3.5-turbo",
model_name: str = "gpt-4o-mini",
agent_names: list[str] = [],
background: ScriptBackground | None = None,
) -> None:
Expand Down
16 changes: 16 additions & 0 deletions docs/pages/concepts/generation.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,8 @@ async def agenerate(
output_parser: BaseOutputParser[OutputType],
temperature: float = 0.7,
structured_output: bool = False,
bad_output_process_model: str = DEFAULT_BAD_OUTPUT_PROCESS_MODEL,
use_fixed_model_version: bool = True
) -> OutputType:
input_variables = re.findall(r"(?<!{){([^{}]+)}(?!})", template)
```
Expand All @@ -23,6 +25,12 @@ The `agenerate` function is versatile by taking the output_parser as an argument
* `gpt-4o-mini-2024-07-18` and later
* `gpt-4o-2024-08-06` and later

The `bad_output_process_model` is used to process the bad output. `DEFAULT_BAD_OUTPUT_PROCESS_MODEL` is set to be `gpt-4o-mini` (At the publication time of Sotopia, we used `gpt-3.5-turbo-0613`. However this model has been taken off the shelf by OpenAI.).

The `use_fixed_model_version` is used to determine whether to use the fixed model version. If set to `True`, the model version will be fixed to the version that was used in Sotopia paper. If set to `False`, the model version will be the latest version available.

Warning: As some fixed model versions might not be available in the OpenAI API, setting `use_fixed_model_version = True` might result in an error.

</Callout>

Here are a few examples of how to use the `agenerate` function:
Expand All @@ -37,6 +45,8 @@ async def agenerate_env_profile(
inspiration_prompt: str = "asking my boyfriend to stop being friends with his ex",
examples: str = "",
temperature: float = 0.7,
bad_output_process_model: str = DEFAULT_BAD_OUTPUT_PROCESS_MODEL,
use_fixed_model_version: bool = True
) -> tuple[EnvironmentProfile, str]:
"""
Using langchain to generate the background
Expand All @@ -56,6 +66,8 @@ async def agenerate_env_profile(
),
output_parser=PydanticOutputParser(pydantic_object=EnvironmentProfile),
temperature=temperature,
bad_output_process_model=bad_output_process_model,
use_fixed_model_version=use_fixed_model_version
)
```
### Other generation functions
Expand All @@ -66,6 +78,8 @@ Similarly, there are other utility functions that builds upon the `agenerate` fu
async def agenerate_relationship_profile(
model_name: str,
agents_profiles: list[str],
bad_output_process_model: str = DEFAULT_BAD_OUTPUT_PROCESS_MODEL,
use_fixed_model_version: bool = True
) -> tuple[RelationshipProfile, str]
```

Expand All @@ -78,5 +92,7 @@ async def agenerate_script(
agent_name: str = "",
history: str = "",
single_step: bool = False,
bad_output_process_model: str = DEFAULT_BAD_OUTPUT_PROCESS_MODEL,
use_fixed_model_version: bool = True
) -> tuple[ScriptInteractionReturnType, str]
```
4 changes: 2 additions & 2 deletions docs/pages/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -206,8 +206,8 @@ asyncio.run(
run_async_server(
model_dict={
"env": "gpt-4",
"agent1": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"agent1": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
},
sampler=UniformSampler(),
)
Expand Down
4 changes: 2 additions & 2 deletions examples/benchmark_evaluator.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@

target_model_patterns: list[list[str]] = [
["gpt-4", "gpt-4", "gpt-3.5-turbo"],
["gpt-4", "gpt-3.5-turbo", "gpt-4"],
["gpt-4", "gpt-3.5-turbo", "togethercomputer/llama-2-70b-chat"],
["gpt-4", "gpt-4o-mini", "gpt-4"],
["gpt-4", "gpt-4o-mini", "togethercomputer/llama-2-70b-chat"],
["gpt-4", "togethercomputer/llama-2-70b-chat", "gpt-3.5-turbo"],
]

Expand Down
4 changes: 2 additions & 2 deletions examples/experiment_eval.py
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ def run_async_server_in_batch(
batch_size: int = 1,
model_names: dict[str, LLM_Name] = {
"env": "gpt-4",
"agent1": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"agent1": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
},
tag: str | None = None,
verbose: bool = False,
Expand Down
4 changes: 2 additions & 2 deletions examples/fix_missing_episodes.py
Original file line number Diff line number Diff line change
Expand Up @@ -252,8 +252,8 @@ def re_run_missing_episodes(
combo_with_models: dict[tuple[LLM_Name, LLM_Name], list[tuple[str, str, str]]],
model_names: dict[str, LLM_Name] = {
"env": "gpt-4",
"agent1": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"agent1": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
},
batch_size: int = 5,
verbose: bool = False,
Expand Down
4 changes: 2 additions & 2 deletions examples/fix_missing_episodes_with_tag.py
Original file line number Diff line number Diff line change
Expand Up @@ -350,8 +350,8 @@ def re_run_missing_episodes(
env_agent_ids: List[Tuple[str, str, str]] = [],
model_names: dict[str, LLM_Name] = {
"env": "gpt-4",
"agent1": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"agent1": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
},
batch_size: int = 5,
rerun_tag: str = "missing_episodes",
Expand Down
2 changes: 1 addition & 1 deletion examples/generate_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ def full_freeform(
def run_async_server_in_batch_script(
*,
batch_size: int = 10,
model: LLM_Name = "gpt-3.5-turbo",
model: LLM_Name = "gpt-4o-mini",
tag: str | None = None,
push_to_db: bool = True,
json_in_script: bool = False,
Expand Down
4 changes: 2 additions & 2 deletions examples/minimalist_demo.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@
run_async_server(
model_dict={
"env": "gpt-4",
"agent1": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"agent1": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
},
sampler=UniformSampler(),
)
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "sotopia"
version = "0.1.0-rc.3"
version = "0.1.0-rc.5"
Copy link
Collaborator

@yufansong yufansong Oct 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why directly junp from 3 to 5 and skip 4 🤔, typo? Seems irreversible but no severe impact.
image

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a typo yeah, my bad

description = "A platform for simulating and evaluating social interaction."
authors = ["Hao Zhu <[email protected]>, Xuhui Zhou <[email protected]>"]
license = "MIT License"
Expand Down
10 changes: 5 additions & 5 deletions sotopia/agents/generate_agent_background.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ async def generate_background(
else:
initial_profile = str(basic_info)
profile = await agenerate_init_profile(
model_name="gpt-3.5-turbo", basic_info=basic_info
model_name="gpt-4o-mini", basic_info=basic_info
)
first_narrative = convert_narratives(
model_name="gpt-3.5-turbo", narrative="first", text=profile
model_name="gpt-4o-mini", narrative="first", text=profile
)
second_narrative = convert_narratives(
model_name="gpt-3.5-turbo", narrative="second", text=profile
model_name="gpt-4o-mini", narrative="second", text=profile
)
previous_messages = []
return (
Expand Down Expand Up @@ -67,8 +67,8 @@ def generate_background_conversation(
json.dump(background_dict, f, indent=4)

model_names: dict[str, str] = {
"env": "gpt-3.5-turbo",
"agent2": "gpt-3.5-turbo",
"env": "gpt-4o-mini",
"agent2": "gpt-4o-mini",
"agent1": "gpt-4",
}

Expand Down
4 changes: 2 additions & 2 deletions sotopia/agents/llm_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ def __init__(
agent_name: str | None = None,
uuid_str: str | None = None,
agent_profile: AgentProfile | None = None,
model_name: str = "gpt-3.5-turbo",
model_name: str = "gpt-4o-mini",
script_like: bool = False,
) -> None:
super().__init__(
Expand Down Expand Up @@ -99,7 +99,7 @@ def __init__(
agent_name: str | None = None,
uuid_str: str | None = None,
agent_profile: AgentProfile | None = None,
model_name: str = "gpt-3.5-turbo",
model_name: str = "gpt-4o-mini",
agent_names: list[str] = [],
background: ScriptBackground | None = None,
) -> None:
Expand Down
2 changes: 1 addition & 1 deletion sotopia/envs/parallel.py
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ def __init__(
["none", "speak", "non-verbal communication", "action", "leave"]
),
action_order: Literal["simultaneous", "round-robin", "random"] = "simultaneous",
model_name: str = "gpt-3.5-turbo",
model_name: str = "gpt-4o-mini",
evaluators: list[Evaluator] = [],
terminal_evaluators: list[Evaluator] = [],
uuid_str: str | None = None,
Expand Down
Loading
Loading