From abc9c9c32fb45a0db8571f0895ba34c98b1e7655 Mon Sep 17 00:00:00 2001 From: Ren Xuancheng <jklj077@users.noreply.github.com> Date: Wed, 30 Oct 2024 10:44:31 +0800 Subject: [PATCH] add vllm version warning (#1043) --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 2667165..4181bb0 100644 --- a/README.md +++ b/README.md @@ -172,6 +172,9 @@ Qwen2.5 is supported by multiple inference frameworks. Here we demonstrate the u ### vLLM +> [!Warning] +> Not working with vllm 0.6.3. + We advise you to use the latest version of vLLM to build OpenAI-compatible API service, including tool use support. Start the server with a chat model, e.g. `Qwen2.5-7B-Instruct`: ```shell vllm serve Qwen/Qwen2.5-7B-Instruct -- GitLab