Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit f6842e8

Browse files
authoredApr 8, 2025··
add related docs for intel cpu/xpu/hpu container (#550)
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
1 parent 3c50308 commit f6842e8

File tree

3 files changed

+113
-0
lines changed

3 files changed

+113
-0
lines changed
 

‎backends/python/server/requirements-intel.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,3 +42,4 @@ win32-setctime==1.1.0 ; python_version >= "3.9" and python_version < "3.13" and
4242
wrapt==1.15.0 ; python_version >= "3.9" and python_version < "3.13"
4343
transformers==4.40.0 ; python_version >= "3.9" and python_version < "3.13"
4444
pyrsistent==0.20.0 ; python_version >= "3.9" and python_version < "3.13"
45+
einops==0.8.0 ; python_version >= "3.9" and python_version < "3.13"

‎docs/source/en/_toctree.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,8 @@
1919
# title: Using TEI CLI
2020
- local: custom_container
2121
title: Build custom container for TEI
22+
- local: intel_container
23+
title: Using TEI container with Intel Hardware
2224
- local: examples
2325
title: Example uses
2426
title: Tutorials

‎docs/source/en/intel_container.md

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
<!--Copyright 2023 The HuggingFace Team. All rights reserved.
2+
3+
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
4+
the License. You may obtain a copy of the License at
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
9+
an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
10+
specific language governing permissions and limitations under the License.
11+
12+
⚠️ Note that this file is in Markdown but contains specific syntax for our doc-builder (similar to MDX) that may not be
13+
rendered properly in your Markdown viewer.
14+
15+
-->
16+
17+
# Using TEI Container with Intel® Hardware
18+
19+
This guide explains how to build and deploy `text-embeddings-inference` containers optimized for Intel® hardware, including CPUs, XPUs, and HPUs.
20+
21+
## CPU
22+
23+
### Build Docker Image
24+
25+
To build a container optimized for Intel® CPUs, run the following command:
26+
27+
```shell
28+
platform="cpu"
29+
30+
docker build . -f Dockerfile-intel --build-arg PLATFORM=$platform -t tei_cpu_ipex
31+
```
32+
33+
### Deploy Docker Container
34+
35+
To deploy your model on an Intel® CPU, use the following command:
36+
37+
```shell
38+
model='BAAI/bge-large-en-v1.5'
39+
volume=$PWD/data
40+
41+
docker run -p 8080:80 -v $volume:/data tei_cpu_ipex --model-id $model
42+
```
43+
44+
## XPU
45+
46+
### Build Docker Image
47+
48+
To build a container optimized for Intel® XPUs, run the following command:
49+
50+
```shell
51+
platform="xpu"
52+
53+
docker build . -f Dockerfile-intel --build-arg PLATFORM=$platform -t tei_xpu_ipex
54+
```
55+
56+
### Deploy Docker Container
57+
58+
To deploy your model on an Intel® XPU, use the following command:
59+
60+
```shell
61+
model='BAAI/bge-large-en-v1.5'
62+
volume=$PWD/data
63+
64+
docker run -p 8080:80 -v $volume:/data --device=/dev/dri -v /dev/dri/by-path:/dev/dri/by-path tei_xpu_ipex --model-id $model --dtype float16
65+
```
66+
67+
## HPU
68+
69+
### Build Docker Image
70+
71+
To build a container optimized for Intel® HPUs (Gaudi), run the following command:
72+
73+
```shell
74+
platform="hpu"
75+
76+
docker build . -f Dockerfile-intel --build-arg PLATFORM=$platform -t tei_hpu
77+
```
78+
79+
### Deploy Docker Container
80+
81+
To deploy your model on an Intel® HPU (Gaudi), use the following command:
82+
83+
```shell
84+
model='BAAI/bge-large-en-v1.5'
85+
volume=$PWD/data
86+
87+
docker run -p 8080:80 -v $volume:/data --runtime=habana -e HABANA_VISIBLE_DEVICES=all -e MAX_WARMUP_SEQUENCE_LENGTH=512 tei_hpu --model-id $model --dtype bfloat16
88+
```
89+
90+
## Prebuilt Docker Images
91+
92+
For convenience, prebuilt Docker images are available on GitHub Container Registry (GHCR). You can pull these images directly without the need to build them manually:
93+
94+
### CPU
95+
To use the prebuilt image optimized for Intel® CPUs, run:
96+
```shell
97+
docker pull ghcr.io/huggingface/text-embeddings-inference:cpu-ipex-latest
98+
```
99+
100+
### XPU
101+
To use the prebuilt image optimized for Intel® XPUs, run:
102+
```shell
103+
docker pull ghcr.io/huggingface/text-embeddings-inference:xpu-ipex-latest
104+
```
105+
106+
### HPU
107+
To use the prebuilt image optimized for Intel® HPUs (Gaudi), run:
108+
```shell
109+
docker pull ghcr.io/huggingface/text-embeddings-inference:hpu-latest
110+
```

0 commit comments

Comments
 (0)
Please sign in to comment.