[Question]: Fail to access model(Ollama/qwen3:32b).

[WangGuangrong]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

RagFlow and Ollama are not on the same server. RagFlow is configured as follows:

The Ollama server's port 11434 has been exposed.

The embedding in the model type can be connected, but both Chat and OCR display: Fail to access model.

What should I do?

[dosubot]

Hi @WangGuangrong! I'm Dosu and I’m helping the ragflow team.

This is a known bug in RAGFlow v0.20.4 that specifically breaks Ollama chat model connections while embedding models continue to work. It was caused by improper URL handling (trailing slashes) in the base_url parameter.

Solutions (pick one):

1. Upgrade to v0.20.5 or newer (recommended) - or use the nightly build (infiniflow/ragflow:nightly). The bug was fixed in PR #9816.

2. Downgrade to v0.20.3 - confirmed to work with remote Ollama.

3. Manual patch - edit rag/llm/__init__.py and add base_url = base_url.rstrip('/').

Also verify these configuration settings:

• API key field: Must have a value (even a dummy like no-key-need) - leaving it empty triggers the Chinese API key error
• Base URL: Use http://192.168.121.182:11434 (no /v1 suffix, no trailing slash)
• Model name: Must exactly match ollama list output (e.g., deepseek-r1:32b)
• Ollama server: Ensure OLLAMA_HOST=0.0.0.0:11434 is set so it listens on all interfaces

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[WangGuangrong]

RagFlow version: 0.24.0，Configuring these lists is done.
But the problem still exists

[dosubot]

The remote Ollama connection bug fix (PR #9816) was merged after v0.24.0, so v0.24.0 still has this issue. The fix specifically addressed chat/OCR models failing to connect to remote Ollama while embedding models worked.

Recommended: Upgrade to a newer version (latest release or infiniflow/ragflow:nightly) to get this fix.

If you can't upgrade immediately, try this debugging from inside the RAGFlow container:

# Test basic connectivity
curl http://192.168.121.182:11434/

# Test chat endpoint directly
curl http://192.168.121.182:11434/api/chat -d '{"model": "deepseek-r1:32b", "messages": [{"role": "user", "content": "hello"}]}'

Also check the full RAGFlow logs (docker logs <ragflow-container>) - the actual exception message after "Fail to access model" will show the specific error (timeout, connection refused, etc.).

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}

   

[Magicbook1108]

You don’t need to fill in the API key field. And just click confirm.

[WangGuangrong]

ragflow-cpu-1 | 22:52:18 - LiteLLM:INFO: utils.py:3748 -
ragflow-cpu-1 | LiteLLM completion() model= deepseek-r1:32b; provider = ollama_chat
ragflow-cpu-1 | 2026-03-05 22:52:18,908 INFO 16
ragflow-cpu-1 | LiteLLM completion() model= deepseek-r1:32b; provider = ollama_chat
ragflow-cpu-1 | 2026-03-05 22:52:28,907 ERROR 16
ragflow-cpu-1 | Fail to access model(Ollama/deepseek-r1:32b).
ragflow-cpu-1 | Traceback (most recent call last):
ragflow-cpu-1 | File "/root/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 520, in wait_for
ragflow-cpu-1 | return await fut
ragflow-cpu-1 | ^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/rag/llm/chat_model.py", line 1224, in async_chat
ragflow-cpu-1 | response = await litellm.acompletion(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/utils.py", line 1759, in wrapper_async
ragflow-cpu-1 | result = await original_function(*args, **kwargs)
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/main.py", line 609, in acompletion
ragflow-cpu-1 | response = await init_response
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 293, in async_completion
ragflow-cpu-1 | response = await self._make_common_async_call(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/llm_http_handler.py", line 161, in _make_common_async_call
ragflow-cpu-1 | response = await async_httpx_client.post(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/litellm_core_utils/logging_utils.py", line 190, in async_wrapper
ragflow-cpu-1 | result = await func(*args, **kwargs)
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 419, in post
ragflow-cpu-1 | response = await self.client.send(req, stream=stream)
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1629, in send
ragflow-cpu-1 | response = await self._send_handling_auth(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
ragflow-cpu-1 | response = await self._send_handling_redirects(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1694, in _send_handling_redirects
ragflow-cpu-1 | response = await self._send_single_request(request)
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1730, in _send_single_request
ragflow-cpu-1 | response = await transport.handle_async_request(request)
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/aiohttp_transport.py", line 274, in handle_async_request
ragflow-cpu-1 | response = await self._make_aiohttp_request(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/litellm/llms/custom_httpx/aiohttp_transport.py", line 240, in _make_aiohttp_request
ragflow-cpu-1 | response = await client_session.request(
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 1510, in aenter
ragflow-cpu-1 | self._resp: _RetType = await self._coro
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 779, in _request
ragflow-cpu-1 | resp = await handler(req)
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/aiohttp/client.py", line 757, in _connect_and_send_request
ragflow-cpu-1 | await resp.start(conn)
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 539, in start
ragflow-cpu-1 | message, payload = await protocol.read() # type: ignore[union-attr]
ragflow-cpu-1 | ^^^^^^^^^^^^^^^^^^^^^
ragflow-cpu-1 | File "/ragflow/.venv/lib/python3.12/site-packages/aiohttp/streams.py", line 703, in read
ragflow-cpu-1 | await self._waiter
ragflow-cpu-1 | File "/root/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/futures.py", line 289, in await
ragflow-cpu-1 | yield self # This tells Task to wait for completion.
ragflow-cpu-1 | ^^^^^^^^^^
ragflow-cpu-1 | File "/root/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/tasks.py", line 385, in __wakeup
ragflow-cpu-1 | future.result()
ragflow-cpu-1 | File "/root/.local/share/uv/python/cpython-3.12.12-linux-x86_64-gnu/lib/python3.12/asyncio/futures.py", line 197, in result
ragflow-cpu-1 | raise self._make_cancelled_error()
ragflow-cpu-1 | asyncio.exceptions.CancelledError
ragflow-cpu-1 | [2026-03-05 22:52:28 +0800] [16] [INFO] 127.0.0.1:51066 POST /v1/llm/add_llm 1.1 200 73 10026731
ragflow-cpu-1 | 2026-03-05 22:52:35,001 INFO 16 [HISTORY][
ragflow-cpu-1 | {
ragflow-cpu-1 | "role": "user",
ragflow-cpu-1 | "content": "Hello! How are you doing!"
ragflow-cpu-1 | }
ragflow-cpu-1 | ]
ragflow-cpu-1 | 22:52:35 - LiteLLM:INFO: utils.py:3748 -
ragflow-cpu-1 | LiteLLM completion() model= deepseek-r1:32b; provider = ollama_chat
ragflow-cpu-1 | 2026-03-05 22:52:35,003 INFO 16
ragflow-cpu-1 | LiteLLM completion() model= deepseek-r1:32b; provider = ollama_chat
ragflow-cpu-1 | 2026-03-05 22:52:40,712 INFO 22 task_executor_e5bb9d7dff38_0 reported heartbeat: {"ip_address": "172.18.0.6", "pid": 22, "name": "task_executor_e5bb9d7dff38_0", "now": "2026-03-05T22:52:40.711+08:00", "boot_at": "2026-03-05T22:08:09.445+08:00", "pending": 1, "lag": 0, "done": 0, "failed": 0, "current": {}}
ragflow-cpu-1 | [2026-03-05 22:52:45 +0800] [16] [INFO] 127.0.0.1:41648 POST /v1/llm/add_llm 1.1 200 116 10015637
ragflow-cpu-1 | 2026-03-05 22:53:10,717 INFO 22 task_executor_e5bb9d7dff38_0 reported heartbeat: {"ip_address": "172.18.0.6", "pid": 22, "name": "task_executor_e5bb9d7dff38_0", "now": "2026-03-05T22:53:10.716+08:00", "boot_at": "2026-03-05T22:08:09.445+08:00", "pending": 1, "lag": 0, "done": 0, "failed": 0, "current": {}}
ragflow-cpu-1 | 2026-03-05 22:53:40,721 INFO 22 task_executor_e5bb9d7dff38_0 reported heartbeat: {"ip_address": "172.18.0.6", "pid": 22, "name": "task_executor_e5bb9d7dff38_0", "now": "2026-03-05T22:53:40.720+08:00", "boot_at": "2026-03-05T22:08:09.445+08:00", "pending": 1, "lag": 0, "done": 0, "failed": 0, "current": {}}

@Magicbook1108 The same applies if the api_key is empty.

[yongtenglei]

Hi, @WangGuangrong

This has been fixed in #13392.

Please add this line to docker/.env to temporarily fix it for now.

LLM_TIMEOUT_SECONDS=600

Let me know if this works for you.

Cheers.

[WangGuangrong]

@yongtenglei success，That was indeed the problem. Thank you.