OPTIONS method is not supported when using sglang with the nextchat client

[jjiwei]

I am using qwen1.5 model, how to use openai compatible api interface in sglang to use
Launch command: python -m sglang.launch_server --model-path Qwen1.5-72B-Chat-GPTQ-Int4 --tp 2 --enable-p2p-check --port 8000 --host 0.0.0.0
Using openai compatible api reported this error, "OPTIONS /v1/chat/completions HTTP/1.1" 405 Method Not Allowed.
thx

[merrymercy]

1. Can you successfully run the example in the README? https://github.com/sgl-project/sglang?tab=readme-ov-file#openai-compatible-api
2. Can you share your code for sending the OAI requests?

[jjiwei]

I'm using the nextchat client to try to connect to sglang's OpenAI-compatible api interface, and I'm using the project's template to get through!
thx

[merrymercy]

From the log, you can see POST method is supported, but OPTIONS method is not supported.
I am not sure why the nextchat client uses this endpoint.
You can either

1. Modify nextchat client to not using OPTIONS method.
2. Help us add the support for OPTIONS here.

   sglang/python/sglang/srt/server.py

   Lines 133 to 140 in f95e661

   	@app.post("/v1/completions")
   	async def openai_v1_completions(raw_request: Request):
   	return await v1_completions(tokenizer_manager, raw_request)
   	@app.post("/v1/chat/completions")
   	async def openai_v1_chat_completions(raw_request: Request):
   	return await v1_chat_completions(tokenizer_manager, raw_request)

   . You can lean how to do this from ChatGPT: https://chatgpt.com/share/6d9e0db7-4c7c-47b1-9a85-ad05b5b43890

[wizd]

Easy walk around by adding a proxy: https://github.com/songquanpeng/one-api

[zhyncs]

May you try the latest release? Thanks! ref #1373