Skip to content

Commit e50e195

Browse files
authored
Update README.md to include ONNX (#507)
1 parent f578c1a commit e50e195

File tree

1 file changed

+8
-5
lines changed

1 file changed

+8
-5
lines changed

README.md

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,7 @@ Ember, GTE and E5. TEI implements many features such as:
5656
[Candle](https://github.com/huggingface/candle)
5757
and [cuBLASLt](https://docs.nvidia.com/cuda/cublas/#using-the-cublaslt-api)
5858
* [Safetensors](https://github.com/huggingface/safetensors) weight loading
59+
* [ONNX](https://github.com/onnx/onnx) weight loading
5960
* Production ready (distributed tracing with Open Telemetry, Prometheus metrics)
6061

6162
## Get Started
@@ -478,7 +479,9 @@ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
478479
Then run:
479480

480481
```shell
481-
# On x86
482+
# On x86 with ONNX backend (recommended)
483+
cargo install --path router -F ort
484+
# On x86 with Intel backend
482485
cargo install --path router -F mkl
483486
# On M1 or M2
484487
cargo install --path router -F metal
@@ -498,11 +501,11 @@ text-embeddings-router --model-id $model --port 8080
498501
sudo apt-get install libssl-dev gcc -y
499502
```
500503

501-
### Cuda
504+
### CUDA
502505

503-
GPUs with Cuda compute capabilities < 7.5 are not supported (V100, Titan V, GTX 1000 series, ...).
506+
GPUs with CUDA compute capabilities < 7.5 are not supported (V100, Titan V, GTX 1000 series, ...).
504507

505-
Make sure you have Cuda and the nvidia drivers installed. NVIDIA drivers on your device need to be compatible with CUDA
508+
Make sure you have CUDA and the nvidia drivers installed. NVIDIA drivers on your device need to be compatible with CUDA
506509
version 12.2 or higher.
507510
You also need to add the nvidia binaries to your path:
508511

@@ -538,7 +541,7 @@ You can build the CPU container with:
538541
docker build .
539542
```
540543

541-
To build the Cuda containers, you need to know the compute cap of the GPU you will be using
544+
To build the CUDA containers, you need to know the compute cap of the GPU you will be using
542545
at runtime.
543546

544547
Then you can build the container with:

0 commit comments

Comments
 (0)