Quickstart-MindIE-LLM

This guide shows how to install UCM with MindIE-LLM support, patch the required MindIE-LLM Python modules, configure mindie_llm, and launch mindie_llm_server with UCM as the KV cache backend on Ascend.

Prerequisites

• MindIE-LLM 2.3.0
• Python >= 3.10
• Ascend runtime/toolkit installed and available in the environment. For details, refer to the MindIE-LLM official documentation.

Step 1: Install UCM

UCM can be installed via Docker.

If you want to build UCM from source code (e.g. for development or customization), see Building and Installing UCM from Source.

Step 2: Prepare the UCM configuration

UCM for MindIE-LLM provides Prefix Cache integration through patched MindIE-LLM modules. You can use the packaged config at ucm/integration/mindie/ucm_config.json or provide your own config file.

Minimal example:

{
  "storage_backends": ["/path/to/kvcache"],
  "mindie_config_path": "/usr/local/lib/python3.11/site-packages/mindie_llm/conf/config.json",
  "block_elem_size": 2
}

Save this file as ucm_config.json and replace the example paths with paths from your environment.

Key notes:

• By default, the patch is applied when mindie_llm is first imported after UCM is installed.
• The hook copies the patched uc_utils.py, unifiedcache_mempool.py, and prefix_cache_plugin.py files into the installed mindie_llm package.
• In the provided Docker image, the patch is already applied during image build, so runtime import uses the patched files directly.

Step 3: Launch MindIE-LLM with UCM

Locate the installed mindie_llm package:

python -c "import mindie_llm, os; print(os.path.dirname(mindie_llm.__file__))"

Then locate conf/config.json under that directory.

Ensure the service user can read the configuration file. For example:

chmod 640 <path-to-mindie_llm>/conf/config.json

In config.json, add or update the kvPoolConfig section under BackendConfig:

"BackendConfig": {
  "kvPoolConfig": {
    "backend": "unifiedcache",
    "configPath": "/path/to/your/ucm_config.json",
    "asyncWrite": true
  }
}

Update config.json with the service IP, port, model path, and any other required MindIE-LLM settings.

Run mindie_llm_server to start the service.

If the following message is displayed, the service has started successfully:

Daemon start success!

After startup, inspect the MindIE-LLM logs to confirm that the unifiedcache backend is loaded successfully.

Troubleshooting

MindIE-LLM code is not patched

• Confirm that UCM_ENABLE_MINDIE=1 was set before installation.
• Confirm that UCM_CXX11_ABI=0 or 1 was set correctly before installation.
• Reinstall UCM.
• Verify that mindie_llm is already installed.

configPath not found

• Use an absolute path.
• Ensure the file is readable by the service user.

Service starts but the UCM backend is not enabled

• Recheck BackendConfig.kvPoolConfig.
• Inspect MindIE-LLM startup logs.