How to run in Docker?

[leftmove]

How can I run this library in Docker? Currently, when I include it in my dependencies, it says it cannot find the source file ggml.h, and that the wheel build failed. What does this mean?

Here is the error.

Using CPython 3.12.7 interpreter at: /usr/local/bin/python
Creating virtual environment at: .venv
Resolved 76 packages in 2.79s
error: Failed to prepare distributions
  Caused by: Failed to download and build `pywhispercpp==1.2.0`
  Caused by: Build backend failed to build wheel through `build_wheel` (exit status: 1)

[stdout]
running bdist_wheel
running build
running build_py
creating build/lib.linux-aarch64-cpython-312/pywhispercpp
copying ./pywhispercpp/utils.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp
copying ./pywhispercpp/_logger.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp
copying ./pywhispercpp/model.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp
copying ./pywhispercpp/__init__.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp
copying ./pywhispercpp/constants.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp
creating build/lib.linux-aarch64-cpython-312/pywhispercpp/examples
copying ./pywhispercpp/examples/__init__.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp/examples
copying ./pywhispercpp/examples/assistant.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp/examples
copying ./pywhispercpp/examples/livestream.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp/examples
copying ./pywhispercpp/examples/main.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp/examples
copying ./pywhispercpp/examples/recording.py -> build/lib.linux-aarch64-cpython-312/pywhispercpp/examples
running build_ext
-- The C compiler identification is GNU 12.2.0
-- The CXX compiler identification is GNU 12.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- pybind11 v2.9.2 
-- Found PythonInterp: /root/.cache/uv/builds-v0/.tmplYIiSU/bin/python (found version "3.12.7")
-- Found PythonLibs: /usr/local/lib/libpython3.12.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- ARM detected
-- Configuring done (0.4s)

[stderr]
CMake Deprecation Warning at pybind11/CMakeLists.txt:8 (cmake_minimum_required):
  Compatibility with CMake < 3.5 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value or use a ...<max> suffix to tell
  CMake that the project does not need compatibility with older versions.


CMake Warning (dev) at pybind11/tools/FindPythonLibsNew.cmake:98 (find_package):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning.

Call Stack (most recent call first):
  pybind11/tools/pybind11Tools.cmake:50 (find_package)
  pybind11/tools/pybind11Common.cmake:206 (include)
  pybind11/CMakeLists.txt:200 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at whisper.cpp/CMakeLists.txt:468 (add_library):
  Cannot find source file:

    ggml.h

  Tried extensions .c .C .c++ .cc .cpp .cxx .cu .mpp .m .M .mm .ixx .cppm
  .ccm .cxxm .c++m .h .hh .h++ .hm .hpp .hxx .in .txx .f .F .for .f77 .f90
  .f95 .f03 .hip .ispc


CMake Error at whisper.cpp/CMakeLists.txt:468 (add_library):
  No SOURCES given to target: whisper


CMake Generate step failed.  Build files cannot be regenerated correctly.
Traceback (most recent call last):
  File "<string>", line 11, in <module>
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/build_meta.py", line 431, in build_wheel
    return _build(['bdist_wheel'])
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/build_meta.py", line 422, in _build
    return self._build_with_temp_dir(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/build_meta.py", line 403, in _build_with_temp_dir
    self.run_setup()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/build_meta.py", line 318, in run_setup
    exec(code, locals())
  File "<string>", line 132, in <module>
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 183, in setup
    return run_commands(dist)
           ^^^^^^^^^^^^^^^^^^
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/core.py", line 199, in run_commands
    dist.run_commands()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 954, in run_commands
    self.run_command(cmd)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/dist.py", line 991, in run_command
    super().run_command(command)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/command/bdist_wheel.py", line 407, in run
    self.run_command("build")
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/dist.py", line 991, in run_command
    super().run_command(command)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/cmd.py", line 316, in run_command
    self.distribution.run_command(command)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/dist.py", line 991, in run_command
    super().run_command(command)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/dist.py", line 973, in run_command
    cmd_obj.run()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/command/build_ext.py", line 98, in run
    _build_ext.run(self)
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 359, in run
    self.build_extensions()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 476, in build_extensions
    self._build_extensions_serial()
  File "/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/setuptools/_distutils/command/build_ext.py", line 502, in _build_extensions_serial
    self.build_extension(ext)
  File "<string>", line 118, in build_extension
  File "/usr/local/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['cmake', '/root/.cache/uv/sdists-v5/pypi/pywhispercpp/1.2.0/qU07NrZq-gfmEEDNyCqAL/pywhispercpp-1.2.0.tar.gz', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/root/.cache/uv/sdists-v5/pypi/pywhispercpp/1.2.0/qU07NrZq-gfmEEDNyCqAL/pywhispercpp-1.2.0.tar.gz/build/lib.linux-aarch64-cpython-312/', '-DPYTHON_EXECUTABLE=/root/.cache/uv/builds-v0/.tmplYIiSU/bin/python', '-DCMAKE_BUILD_TYPE=Release', '-DEXAMPLE_VERSION_INFO=1.2.0', '-GNinja', '-DCMAKE_MAKE_PROGRAM:FILEPATH=/root/.cache/uv/builds-v0/.tmplYIiSU/lib/python3.12/site-packages/ninja/data/bin/ninja']' returned non-zero exit status 1.
The command '/bin/sh -c uv sync' returned a non-zero code: 2

Here is my Dockerfile.

FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim

WORKDIR /app

COPY pyproject.toml /app/pyproject.toml
COPY main.py /app/main.py
COPY routers /app/routers
COPY prompts /app/prompts
COPY outputs /app/outputs
COPY audio /app/audio

RUN apt update
RUN apt install -y build-essential ffmpeg
RUN uv sync

COPY .env /app/.env
COPY env.py /app/env.py

EXPOSE ${PORT}

CMD ["uv", "run", "python", "main.py"]

I use uv if that's relevant. Here is the dependencies section of my pyproject.toml.

dependencies = [
    "fastapi>=0.115.0",
    "google-generativeai>=0.8.2",
    "jsonpickle>=3.3.0",
    "more-itertools==10.5.0",
    "numba==0.60.0",
    "numpy==2.0.2",
    "pydub>=0.25.1",
    "python-dotenv>=1.0.1",
    "python-multipart>=0.0.12",
    "pywhispercpp>=1.2.0",
    "regex==2024.9.11",
    "soundfile>=0.12.1",
    "supabase>=2.8.1",
    "tiktoken==0.8.0",
    "uvicorn>=0.31.0",
]

[absadiki]

@leftmove, it seems like you are using an ARM machine. There are no pre-built wheels for arm in the current PYPI version unfortunately.
I'll try to include it in the next release.
In the meantime, You can try to build it from the source.

[akamick86]

Would be really great to have a wheel for ARM Linux.

[absadiki]

@akamick86, I tried, but it turns out that GitHub does not offer ARM instances in the free plan, unfortunately.

[zhemaituk]

@absadiki It seems that ARM used to be on paid plans only, but not anymore since recently. Github actions doc says "The use of these runners on public repositories is free and unlimited.":
https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
Maybe worth a try? Before that people used to use docker buildx for multi platform/arch builds, but that's probably more work.

[absadiki]

Thanks @zhemaituk for letting me know! I wasn’t aware of that until now.
Yes, I’ll give it another try soon.

[absadiki]

@zhemaituk, @akamick86, The Github action on the new arm instance went successfully.
Could you please give it a test if you have access to an arm machine ?

[zhemaituk]

@absadiki it installs, but there is some problem running it.

test.Dockerfile (you should be able to run on amd machine as well with docker buildx):

FROM --platform=linux/arm64 public.ecr.aws/docker/library/python:3.13

RUN python -m pip install --upgrade  \
    pip \
    setuptools \
    wheel

# RUN apt-get update && apt-get install -y cmake
# ENV CMAKE_ARGS="-DCMAKE_POLICY_VERSION_MINIMUM=3.5"
# RUN pip install pywhispercpp

ARG WHEEL="pywhispercpp-0.1.dev1+gd1cfbd6-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl"
COPY ${WHEEL} ./
RUN pip install ${WHEEL}

COPY file.wav ./
COPY <<"EOT" test.py
from pywhispercpp.model import Model
model = Model("base.en")
segments = model.transcribe("file.wav")
for segment in segments:
  print(segment.text)
EOT

RUN python test.py

It fails running:

5.588 whisper_init_from_file_with_params_no_state: loading model from '/root/.local/share/pywhispercpp/models/ggml-base.en.bin'
5.588 whisper_init_with_params_no_state: use gpu    = 1
5.588 whisper_init_with_params_no_state: flash attn = 0
5.588 whisper_init_with_params_no_state: gpu_device = 0
5.588 whisper_init_with_params_no_state: dtw        = 0
5.591 whisper_init_with_params_no_state: devices    = 1
5.591 whisper_init_with_params_no_state: backends   = 1
5.591 whisper_model_load: loading model
5.591 whisper_model_load: n_vocab       = 51864
5.591 whisper_model_load: n_audio_ctx   = 1500
5.591 whisper_model_load: n_audio_state = 512
5.591 whisper_model_load: n_audio_head  = 8
5.591 whisper_model_load: n_audio_layer = 6
5.591 whisper_model_load: n_text_ctx    = 448
5.591 whisper_model_load: n_text_state  = 512
5.591 whisper_model_load: n_text_head   = 8
5.591 whisper_model_load: n_text_layer  = 6
5.591 whisper_model_load: n_mels        = 80
5.591 whisper_model_load: ftype         = 1
5.591 whisper_model_load: qntvr         = 0
5.591 whisper_model_load: type          = 2 (base)
5.706 whisper_model_load: adding 1607 extra tokens
5.707 whisper_model_load: n_langs       = 99
5.709 whisper_model_load:      CPU total size =   147.37 MB
5.945 whisper_model_load: model size    =  147.37 MB
5.950 whisper_init_state: kv self size  =    6.29 MB
5.964 whisper_init_state: kv cross size =   18.87 MB
5.968 whisper_init_state: kv pad  size  =    3.15 MB
5.986 whisper_init_state: compute buffer (conv)   =   16.26 MB
5.993 whisper_init_state: compute buffer (encode) =   85.86 MB
6.002 whisper_init_state: compute buffer (cross)  =    4.65 MB
6.011 whisper_init_state: compute buffer (decode) =   96.35 MB
6.054 Progress:   0%
6.101 Illegal instruction

[absadiki]

@zhemaituk, I also tested it locally using Docker and it worked, the test suite on the github action passed without any issues.

I am not sure, but could you try using an earlier Python version (maybe 3.11)? It seems that using versions 3.12 and 3.13 is causing issues.

[zhemaituk]

Thanks for checking @absadiki .
Unfortunately I was not able to make it work in my env. I tried using the wheel on python3.11, python3.12, python3.13, ubuntu-24.04-arm, manylinux2014_aarch64 docker images running on Apple M3 ARM host. Same problem, the process exits with "Illegal instruction".

Since it works in your env and in github actions, I think it's worth releasing it. I'll also give it a shot, maybe installing from PyPI will somehow work, but if it doesn't work for me I can stick to 1.3.0 for now.

[absadiki]

Thanks a lot, @zhemaituk, for taking the time to test it. To be honest, I have no idea what might be causing this "illegal instruction" issue.

For now, I'll release a new version and push it to PyPI, hopefully, it will be tested by others as well.

[zhemaituk]

@absadiki PyPI didn't do the magic for me :).
It seems rather related to whisper.cpp running/building in docker on Apple ARM host in the latest versions. As I also tried to build from sources - and whisper.cpp v1.7.4/v1.7.5 build fails (/usr/lib/gcc/aarch64-linux-gnu/12/include/arm_neon.h:28760:1: error: inlining failed in call to 'always_inline' 'vaddq_f16'). There are a couple of similar issues reported to whisper.cpp, possibly it will be addressed in future versions.
v1.6.2 builds and works fine, the one used in 1.3.0 wrapper.

In either case thank you for adding arm support!

I will leave this here in case someone encounters the same errors, this what worked for me to build on Apple ARM in Docker:

FROM python:3.13

RUN apt-get update && apt-get install -y cmake
ENV CMAKE_ARGS="-DCMAKE_POLICY_VERSION_MINIMUM=3.5"
RUN pip install pywhispercpp==1.3.0

[absadiki]

Thanks @zhemaituk for the detailed info, super helpful!
That makes sense, and it's great to know that 1.6.2 works in that setup.
Really appreciate you sharing this for others who might run into the same issue.
Hopefully, this gets resolved in future versions!

[zhemaituk]

Here you go! Works after:
export CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv8-a"

@absadiki it may make sense to include those flags for arm build, so the binaries in PyPI are compatible with more architectures. This is exactly what whisper.cpp is doing in their pipelines: https://github.com/ggml-org/whisper.cpp/blob/78b31ca7824500e429ba026c1a9b48e0b41c50cb/.github/workflows/build.yml#L134

FROM python:3.13

RUN apt-get update && apt-get install -y cmake ffmpeg

ENV CMAKE_ARGS="-DGGML_NATIVE=OFF -DGGML_CPU_ARM_ARCH=armv8-a"
RUN pip install pywhispercpp --no-binary pywhispercpp