Support multiple stop options for OpenAI models

[simonw]

llm/llm/default_plugins/openai_models.py

Lines 107 to 110 in 898418a

	stop: Optional[str] = Field(
	description=("A string where the API will stop generating further tokens."),
	default=None,
	)

Ideally this would duplicate the official docs:

Up to 4 sequences where the API will stop generating further tokens.

[simonw]

The trick here is figuring out the syntax, since options can only be specified once as a string. I'll probably do a trick a bit like what I did with logit_bias:

llm/llm/default_plugins/openai_models.py

Lines 118 to 141 in 898418a

	@field_validator("logit_bias")
	def validate_logit_bias(cls, logit_bias):
	if logit_bias is None:
	return None
	if isinstance(logit_bias, str):
	try:
	logit_bias = json.loads(logit_bias)
	except json.JSONDecodeError:
	raise ValueError("Invalid JSON in logit_bias string")
	validated_logit_bias = {}
	for key, value in logit_bias.items():
	try:
	int_key = int(key)
	int_value = int(value)
	if -100 <= int_value <= 100:
	validated_logit_bias[int_key] = int_value
	else:
	raise ValueError("Value must be between -100 and 100")
	except ValueError:
	raise ValueError("Invalid key-value pair in logit_bias dictionary")
	return validated_logit_bias

[simonw]

Here's how Replicate partially solved this: https://replicate.com/joehoover/falcon-40b-instruct

Comma delimited string specifying stop sequences. Multi-token stop sequences are supported, but they cannot contain commas.