-
Notifications
You must be signed in to change notification settings - Fork 263
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Minor fixes to ds_tool and infer_tool #36
Conversation
juberti
commented
Jun 23, 2024
- --upload_split param to allow the dest split to be different than the src split
- allow @file syntax for --prompt
- add retries and timeouts to TTS requests
- --upload_split param to allow the dest split to be different than the src split - allow @file syntax for --prompt - add retries and timeouts to TTS requests
@@ -150,6 +159,7 @@ def main(args: DatasetToolArgs): | |||
"token": token, | |||
"revision": args.upload_branch, | |||
"private": args.private, | |||
"split": args.upload_split, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Wait, you didn't fix the conflict here!
Now on line 170 you'll be sending the split
argument twice, and with different values!
assert ( | ||
not self.upload_split or self.dataset_split | ||
), "Must specify dataset_split when using upload_split" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I didn't catch this the first time, but why can't upload_split
be equal to dataset_split
by default if not specified?
* Minor fixes to ds_tool and infer_tool - --upload_split param to allow the dest split to be different than the src split - allow @file syntax for --prompt - add retries and timeouts to TTS requests * docs
If max_response_tokens is set (by default 50), the number of response tokens is truncated to this number. This helps avoid situations in which a very long response cause OOM issues. See this comparison on the choice of default value: https://3.basecamp.com/5478728/buckets/36374248/todos/8256002600