Skip to content

Commit 6c33ec9

Browse files
authored
update vllm-ipex, boost servi performance (#1941)
1 parent 9c6cd5a commit 6c33ec9

File tree

4 files changed

+8
-8
lines changed

4 files changed

+8
-8
lines changed

comps/llms/deployment/docker_compose/compose_text-generation.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -186,7 +186,7 @@ services:
186186
ZE_AFFINITY_MASK: ${ZE_AFFINITY_MASK}
187187
shm_size: 128g
188188
entrypoint: /bin/bash -c "\
189-
chmod +x /llm/vllm_ipex_entrypoint.sh && \
189+
source /opt/intel/oneapi/setvars.sh --force && \
190190
bash /llm/vllm_ipex_entrypoint.sh"
191191

192192
networks:

comps/llms/src/text-generation/README_vllm_ipex.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ This service provides high-throughput, low-latency LLM serving accelerated by vL
1919
You must download the official docker image from [Docker Hub](https://hub.docker.com/r/intel/llm-scaler-vllm) first.
2020

2121
```bash
22-
docker pull intel/llm-scaler-vllm:1.0
22+
docker pull intel/llm-scaler-vllm:0.10.0-b4
2323
```
2424

2525
## Start Microservice
@@ -31,9 +31,9 @@ Deploy the vLLM-IPEX model serving using Docker Compose.
3131
1. Export the required environment variables:
3232

3333
```bash
34-
# Use image: intel/llm-scaler-vllm:1.0
34+
# Use image: intel/llm-scaler-vllm:0.10.0-b4
3535
export REGISTRY=intel
36-
export TAG=1.0
36+
export TAG=0.10.0-b4
3737

3838
export VIDEO_GROUP_ID=$(getent group video | awk -F: '{printf "%s\n", $3}')
3939
export RENDER_GROUP_ID=$(getent group render | awk -F: '{printf "%s\n", $3}')

comps/lvms/deployment/docker_compose/compose.yaml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -188,7 +188,7 @@ services:
188188
ZE_AFFINITY_MASK: ${ZE_AFFINITY_MASK}
189189
shm_size: 128g
190190
entrypoint: /bin/bash -c "\
191-
chmod +x /llm/vllm_ipex_entrypoint.sh && \
191+
source /opt/intel/oneapi/setvars.sh --force && \
192192
bash /llm/vllm_ipex_entrypoint.sh"
193193

194194
networks:

comps/lvms/src/README_vllm_ipex.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ This service provides high-throughput, low-latency LVM serving accelerated by vL
1919
You must download the official docker image from [Docker Hub](https://hub.docker.com/r/intel/llm-scaler-vllm) first.
2020

2121
```bash
22-
docker pull intel/llm-scaler-vllm:1.0
22+
docker pull intel/llm-scaler-vllm:0.10.0-b4
2323
```
2424

2525
## Start Microservice
@@ -31,9 +31,9 @@ Deploy the vLLM-IPEX model serving using Docker Compose.
3131
1. Export the required environment variables:
3232

3333
```bash
34-
# Use image: intel/llm-scaler-vllm:1.0
34+
# Use image: intel/llm-scaler-vllm:0.10.0-b4
3535
export REGISTRY=intel
36-
export TAG=1.0
36+
export TAG=0.10.0-b4
3737

3838
export ip_address=$(hostname -I | awk '{print $1}')
3939
export VIDEO_GROUP_ID=$(getent group video | awk -F: '{printf "%s\n", $3}')

0 commit comments

Comments
 (0)