Update README.md
Browse files
README.md
CHANGED
|
@@ -57,6 +57,13 @@ print(responses)
|
|
| 57 |
```
|
| 58 |
|
| 59 |
### Inference with [vLLM](https://github.com/vllm-project/vllm)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
```python
|
| 61 |
from transformers import AutoTokenizer
|
| 62 |
from vllm import LLM, SamplingParams
|
|
|
|
| 57 |
```
|
| 58 |
|
| 59 |
### Inference with [vLLM](https://github.com/vllm-project/vllm)
|
| 60 |
+
|
| 61 |
+
For now, you need to install our forked version of vLLM.
|
| 62 |
+
|
| 63 |
+
```bash
|
| 64 |
+
pip install git+https://github.com/OpenBMB/vllm.git@minicpm3
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
```python
|
| 68 |
from transformers import AutoTokenizer
|
| 69 |
from vllm import LLM, SamplingParams
|