Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
mtian8 authored Jul 24, 2024
1 parent bc59897 commit 24760e7
Showing 1 changed file with 9 additions and 1 deletion.
10 changes: 9 additions & 1 deletion eval/scripts/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,26 @@ ANTHROPIC_KEY = 'your_api_key'
GOOGLE_KEY = 'your_api_key' 
```

For example, to create model results with `gpt-4o` and the default settings, go to the root of this repo and run
For example, to create model results with `gpt-4o` and the default settings, go to the root of this repo and run

```bash
python eval/scripts/gencode_json.py --model gpt-4o
```

For results with scientist-annotated background, run

```bash
python eval/scripts/gencode_json.py --model gpt-4o --with-background
```


### Command-Line Arguments

- `--model` - Specifies the model name used for generating responses.
- `--output-dir` - Directory to store the generated code outputs (Default: `eval_results/generated_code`).
- `--input-path` - Directory containing the JSON files describing the problems (Default: `eval/data/problems_all.jsonl`).
- `--prompt-dir` - Directory where prompt files are saved (Default: `eval_results/prompt`).
- `--with-background` - Include problem background if enabled.
- `--temperature` - Controls the randomness of the generation (Default: 0).

## **Evaluate generated code**
Expand Down

0 comments on commit 24760e7

Please sign in to comment.