Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[TextGeneration] Samling arguments for generation #1225

Merged
merged 15 commits into from
Sep 15, 2023
Merged

[TextGeneration] Samling arguments for generation #1225

merged 15 commits into from
Sep 15, 2023

Conversation

horheynm
Copy link
Member

@horheynm horheynm commented Sep 1, 2023 •
edited
Loading

  1. Add sampling into the text-generation pipeline
  2. Refactor token generation flow

I also need #1234 to land to check on failures caused by this pr if any

dbogunowicz
dbogunowicz suggested changes Sep 7, 2023
View reviewed changes
Copy link
Contributor

@dbogunowicz dbogunowicz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Designwise looks great. Let's clean up the code, make it nicer in terms of style. Also, before landing let's make sure that all the tests in the new LLM testing harness pass:

#1216

This PR has potential to accidently break main pathways.

src/deepsparse/transformers/engines/nl_decoder_engine.py Show resolved Hide resolved
src/deepsparse/transformers/pipelines/text_generation.py Show resolved Hide resolved
src/deepsparse/transformers/utils/token_generator.py Show resolved Hide resolved
src/deepsparse/transformers/pipelines/text_generation.py Outdated Show resolved Hide resolved
src/deepsparse/transformers/pipelines/text_generation.py Show resolved Hide resolved
src/deepsparse/transformers/pipelines/token_classification.py Show resolved Hide resolved
src/deepsparse/transformers/utils/token_generator.py Outdated Show resolved Hide resolved
src/deepsparse/transformers/utils/token_generator.py Outdated Show resolved Hide resolved
src/deepsparse/transformers/utils/token_generator.py Outdated Show resolved Hide resolved
@dbogunowicz
Copy link
Contributor

Also, PR description missing, let's make the PR title more descriptive.

@dbogunowicz dbogunowicz mentioned this pull request Sep 8, 2023
Closed
@horheynm horheynm marked this pull request as ready for review September 11, 2023 20:41
bfineran
bfineran previously approved these changes Sep 12, 2023
View reviewed changes
dbogunowicz
dbogunowicz previously approved these changes Sep 13, 2023
View reviewed changes
src/deepsparse/transformers/pipelines/text_generation.py Outdated Show resolved Hide resolved
@bfineran bfineran dismissed stale reviews from dbogunowicz and themself via 816c5ea September 13, 2023 13:13
@bfineran bfineran changed the title Sampling [TextGeneration] Samling arguments for generation Sep 15, 2023
@bfineran
Copy link
Contributor

tests unrelated, merging

@bfineran bfineran merged commit 55307ad into main Sep 15, 2023
@bfineran bfineran deleted the sampling branch September 15, 2023 14:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mgoin mgoin mgoin left review comments

@bfineran bfineran bfineran approved these changes

@dbogunowicz dbogunowicz dbogunowicz left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@horheynm @dbogunowicz @bfineran @mgoin