Unable to use inference server: Illegal instruction

[herobrine99dan]

Search before asking

• I have searched the Inference issues and found no similar bug report.

Bug

Hello, I was following the instructions to install Inference on a raspberry pi 4, I've got no issues installing all the packages with pip install inference and setting up the docker container, nevertheless after the server starts the docker container exits a few seconds later and the log i'm getting with docker logs is

/tmp/matplotlib is not a writable directory
Matplotlib created a temporary cache directory at /tmp/matplotlib-nbs311ay because there was an issue with the default path (/tmp/matplotlib); it is highly recommended to set the MPLCONFIGDIR environment variable to a writable directory, in particular to speed up the import of Matplotlib and to better support multiprocessing.
Illegal instruction

Environment

Inference: v0.44.1
OS: Debian GNU/Linux 12 (bookworm) aarch64
Device: raspberry pi 4
Python: 3.11.2

Minimal Reproducible Example

i'm following the official guide:
pip install inference-cli
inference server start

Additional

I've tried to install Inference on a raspberry pi 4, and then i've tried a clean installation on a raspberry pi 400, getting the same issue in both cases, i've also tried using an older version of docker. Nothing helped. Everything is up to date.

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[PawelPeczek-Roboflow]

hi there - thanks for raising the issue. The problem is likely not with your setup, but with the inference server itself. I must admit I've never attempted to run on raspbery, so no clue what may be wrong.

That may be not that easy, but could u try to build the image from repository source using the following command:

docker build -t roboflow/roboflow-inference-server-cpu:test -f ./docker/dockerfiles/Dockerfile.onnx.cpu .

[PawelPeczek-Roboflow]

I am under impression that raspbery arch. may require dedicated build, but I am not sure.

[herobrine99dan]

It's strange, because in the docs it says Inference works on Raspberry Pi 4 Model B and Raspberry Pi 5 so long as you are using the 64-bit version of the operating system (if your SD Card is big enough, we recommend the 64-bit "Raspberry Pi OS with desktop and recommended software" version).
Trying to build the image gives me this

34.10 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
34.63 rm -f dist/*
34.63 rm -rf build/*
34.64 python .release/pypi/inference.core.setup.py bdist_wheel
43.65 make: *** [Makefile:40: create_wheels] Illegal instruction (core dumped)
------

 2 warnings found (use docker --debug to expand):
 - FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 1)
 - JSONArgsRecommended: JSON arguments recommended for ENTRYPOINT to prevent unintended behavior related to OS signals (line 81)
Dockerfile.onnx.cpu:54
--------------------
  52 |     WORKDIR /build
  53 |     COPY . .
  54 | >>> RUN make create_wheels
  55 |     RUN pip3 install dist/inference_cli*.whl dist/inference_core*.whl dist/inference_cpu*.whl dist/inference_sdk*.whl "setuptools<=75.5.0"
  56 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c make create_wheels" did not complete successfully: exit code: 2

[PawelPeczek-Roboflow]

well - not sure why this happened, we need to debug internally.

[herobrine99dan]

In the mean time, what are your suggestions to deploy a roboflow object detection on a raspberry? is there an easy guide i can follow without difficulties?

[hansent]

@herobrine99dan could you try to pip install inference. And then run import inference in a python session? Trying to narrow down if it's a issue with our docker images/build or inference code itself.

you may need to apt install a couple of things. The docker installs these:

apt update -y && apt install -y \
    libxext6 \
    libopencv-dev \
    uvicorn \
    python3-pip \
    git \
    libgdal-dev \
    cmake \
    rustc \
    && rm -rf /var/lib/apt/lists/*

[herobrine99dan]

I've installed the stuff you told me, running import inference in a python3 shell doesn't give any issue.