[Misc][Utils] allow get_open_port to be called for multiple times

[youkaichao]

#4914 allows users to specify a port to use, but is restricted to one port. When we want to use multiple ports (e.g. multiple LLM instances in offline inference for interactive playing), it will cause a problem.

This PR makes it possible to call get_open_port multiple times, and return different ports. In a safety-constrained system, the admin can open a port range for vLLM to use.

[DarkLight1337]

Should we really make this the default behaviour? It may be a bit confusing when the port number does not match what was originally specified in the command. Perhaps we should require another env flag to enable this. If get_open_port is called multiple times without this env flag being set, we can log a message to inform the user about this option.

[youkaichao]

It may be a bit confusing when the port number does not match what was originally specified in the command. Perhaps we should require another env flag to enable this.

The first port will be the specified port, but the second port will not.

We cannot achieve this via flags. Users can create arbitary number of LLMs, and each one can require one port.

In addition, vLLM itself might also use more than one port. Currently some code avoids this by using some other protocols (e.g. shared files in

vllm/vllm/distributed/device_communicators/custom_all_reduce_utils.py

Line 115 in ccdc490

init_method = f"file://{temp_path}"

)

This is the best approach I can come up with. After all, users just want to control the port range vLLM uses, but will not limit vLLM to use just one port.

[youkaichao]

@DarkLight1337 I added some logging so that users will know new ports are used.

[DarkLight1337]

In that case, I guess it's fine as long as you inform the user.