Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Add a function to convert sampling_params to kwargs #1170

Merged
merged 10 commits into from
Aug 21, 2024
Merged

[Feature] Add a function to convert sampling_params to kwargs #1170

merged 10 commits into from
Aug 21, 2024

Conversation

gryffindor-rr
Copy link
Contributor

@gryffindor-rr gryffindor-rr commented Aug 21, 2024 •
edited
Loading

Motivation

Modifications

add a function to convert params to a dict to adapt different arg names.

Checklist

  • Format your code according to the Contributor Guide.
  • Add unit tests as outlined in the Contributor Guide.
  • Update documentation as needed, including docstrings or example tutorials.

@JianyuZhan
Copy link

JianyuZhan commented Aug 21, 2024 •
edited
Loading

I also mentioned this circuitous transfrom from SamplingParams class to SamplingParams dict and then to SamplingParams class in my pending PR. I also added a to_dict transform method. I think the better way is when SamplingParams is promoted as a publically accessible API, then do away this cubesome transform.

zhyncs
zhyncs reviewed Aug 21, 2024
View reviewed changes
python/sglang/srt/sampling_params.py
def to_srt_kwargs(self):
return {
"max_new_tokens": self.max_new_tokens,
"stop": self.stop_strs,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does stop_strs need to be processed as a list, in the case of multiple stop str scenarios?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

stop is either str or list of str, both type can be handled by json.dumps. the token_ids is a set which need to be converted to list.

@merrymercy merrymercy merged commit d6aeb9f into sgl-project:main Aug 21, 2024
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zhyncs zhyncs zhyncs left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@gryffindor-rr @JianyuZhan @zhyncs @merrymercy