Getting Started

UCM integrates with the following inference engines by plugging in a KV connector / patched store at runtime. Each engine has its own quick start guide covering installation, configuration, and launching inference.

Which guide should I follow?

Pick the guide that matches the engine you use. All guides assume UCM is already installed; if it is not, see the Installation guide first.

• vLLM (CUDA)

  ---

  Run UCM with vLLM on GPU (CUDA) platforms, including prefix caching and sparse attention. Docker, pip, and runtime integration steps.

  vLLM quick start

• vLLM-Ascend (NPU)

  ---

  Run UCM with vLLM-Ascend on Ascend NPU platforms, including the upgrade deployment steps for the Ascend environment.

  vLLM-Ascend quick start

• SGLang (CUDA)

  ---

  Run UCM with SGLang on GPU (CUDA) platforms via the hierarchical cache configuration.

  SGLang quick start

• MindIE (NPU Ascend)

  ---

  Run UCM with MindIE-LLM on Ascend NPU platforms, including MindIE-LLM patching and mindie_llm_server configuration.

  MindIE quick start

Building from source

If you prefer to build UCM from source code rather than using Docker images or pip wheels, see Building and Installing UCM from Source.