You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
We have some upcoming work, such as annotations, that requires structured output from vLLM. Here's an example of the type of vLLM parameters we need to be able to pass from a pipeline into the vLLM backend to enable guided decoding. The key part is the guided_choice section there, where we need to ensure that gets passed into vLLM to enable guided_choice decoding as documented at https://docs.vllm.ai/en/latest/features/structured_outputs.html
This is similar to #481, but I decided on a narrow new issue here just to ensure a user can choose to use guided decoding in their pipeline. 481 is a bit broader in scope, including actually using it within SDG as opposed to just enabling a user to create a pipeline that uses guided decoding.
Is your feature request related to a problem? Please describe.
We have some upcoming work, such as annotations, that requires structured output from vLLM. Here's an example of the type of vLLM parameters we need to be able to pass from a pipeline into the vLLM backend to enable guided decoding. The key part is the
guided_choice
section there, where we need to ensure that gets passed into vLLM to enable guided_choice decoding as documented at https://docs.vllm.ai/en/latest/features/structured_outputs.htmlDescribe the solution you'd like
Pipelines defined similar to above should cause vLLM to enable its guided_choice to coerce the responses into one of the defined possibilities.
The text was updated successfully, but these errors were encountered: