Skip to content

Commit b16de35

Browse files
[MP]add the lmcache_mp_connector for dev (LMCache#3235)
* add the latest lmcache_mp_connector for dev Signed-off-by: idellzheng <idellzheng@tencent.com> * check style fix Signed-off-by: idellzheng <idellzheng@tencent.com> * update user docs Signed-off-by: idellzheng <idellzheng@tencent.com> * check fix Signed-off-by: idellzheng <idellzheng@tencent.com> --------- Signed-off-by: idellzheng <idellzheng@tencent.com>
1 parent d16c9aa commit b16de35

2 files changed

Lines changed: 1226 additions & 0 deletions

File tree

docs/source/getting_started/quickstart.rst

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -58,6 +58,30 @@ This guide helps you get LMCache running end-to-end in a couple of minutes. Use
5858
--port 8000 --kv-transfer-config \
5959
'{"kv_connector":"LMCacheMPConnector", "kv_role":"kv_both"}'
6060
61+
.. note::
62+
**Where does** ``LMCacheMPConnector`` **resolve to?** This depends on your vLLM version:
63+
64+
- **vLLM < 0.20.0** -- ``"kv_connector":"LMCacheMPConnector"`` always
65+
resolves to vLLM's built-in
66+
``vllm.distributed.kv_transfer.kv_connector.v1.LMCacheMPConnector``;
67+
there is no way to redirect it to the LMCache-shipped implementation.
68+
69+
- **vLLM >= 0.20.0** -- ``"kv_connector":"LMCacheMPConnector"`` still
70+
defaults to vLLM's built-in connector, but you can opt in to the
71+
LMCache-shipped implementation
72+
(:mod:`lmcache.integration.vllm.lmcache_mp_connector`) by adding
73+
``kv_connector_module_path``:
74+
75+
.. code-block:: bash
76+
77+
vllm serve Qwen/Qwen3-8B \
78+
--port 8000 --kv-transfer-config \
79+
'{"kv_connector":"LMCacheMPConnector", "kv_connector_module_path":"lmcache.integration.vllm.lmcache_mp_connector", "kv_role":"kv_both"}'
80+
81+
The LMCache-shipped connector tracks the latest LMCache server
82+
protocol and ships fixes/features ahead of the version vendored
83+
into vLLM, so prefer it whenever you are on vLLM 0.20.0 or newer.
84+
6185
**Test** -- open a new terminal and send two requests whose
6286
prompts share a prefix:
6387

0 commit comments

Comments
 (0)