Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot connect to Ollama server #19

Open
LeGeRyChEeSe opened this issue May 26, 2024 · 3 comments
Open

Cannot connect to Ollama server #19

LeGeRyChEeSe opened this issue May 26, 2024 · 3 comments

Comments

@LeGeRyChEeSe
Copy link

Hello,

I can't connect discollama to my ollama server host at 127.0.0.1:11434 address.

Here is my error message when running the bot with docker and then pinging it on discord with 'Hello' :

2024-05-26 04:30:06 discollama-1  | 2024-05-26 02:30:06 INFO     discord.client logging in using static token
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.668 # <redisgears_2> could not initialize RedisAI_InitError
2024-05-26 04:30:07 discollama-1  | 2024-05-26 02:30:07 INFO     discord.gateway Shard ID None has connected to Gateway (Session ID: XXXXXXXXXXX).
2024-05-26 04:30:05 redis-1       | 
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.668 * <redisgears_2> Failed loading RedisAI API.
2024-05-26 04:30:09 discollama-1  | 2024-05-26 02:30:09 INFO     discord.discollama Ready! Invite URL: XXXXXXXXXXXXXX0
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.668 * <redisgears_2> RedisGears v2.0.20, sha='XXXXXXXXXXXXXXXXXXX', build_type='release', built_for='Linux-ubuntu22.04.x86_64', redis_version:'7.2.4', enterprise:'false'.
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.686 * <redisgears_2> Registered backend: js.
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.693 * Module 'redisgears_2' loaded from /opt/redis-stack/lib/redisgears.so
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.694 * Server initialized
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.695 * Ready to accept connections tcp
2024-05-26 04:30:32 discollama-1  | 2024-05-26 02:30:32 ERROR    discord.client Ignoring exception in on_message
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 170, in try_connect
2024-05-26 04:30:32 discollama-1  |     stream = await asynclib.connect_tcp(remote_host, remote_port, local_address)
2024-05-26 04:30:32 discollama-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2258, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     await get_running_loop().create_connection(
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1085, in create_connection
2024-05-26 04:30:32 discollama-1  |     raise exceptions[0]
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1069, in create_connection
2024-05-26 04:30:32 discollama-1  |     sock = await self._connect_sock(
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 973, in _connect_sock
2024-05-26 04:30:32 discollama-1  |     await self.sock_connect(sock, address)
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 634, in sock_connect
2024-05-26 04:30:32 discollama-1  |     return await fut
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 674, in _sock_connect_cb
2024-05-26 04:30:32 discollama-1  |     raise OSError(err, f'Connect call failed {address}')
2024-05-26 04:30:32 discollama-1  | ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 11434)
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | The above exception was the direct cause of the following exception:
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
2024-05-26 04:30:32 discollama-1  |     yield
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     stream: anyio.abc.ByteStream = await anyio.connect_tcp(
2024-05-26 04:30:32 discollama-1  |                                    ^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 232, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     raise OSError("All connection attempts failed") from cause
2024-05-26 04:30:32 discollama-1  | OSError: All connection attempts failed
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | The above exception was the direct cause of the following exception:
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
2024-05-26 04:30:32 discollama-1  |     yield
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 366, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     resp = await self._pool.handle_async_request(req)
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 268, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     raise exc
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 251, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     response = await connection.handle_async_request(request)
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     raise exc
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     stream = await self._connect(request)
2024-05-26 04:30:32 discollama-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection.py", line 124, in _connect
2024-05-26 04:30:32 discollama-1  |     stream = await self._network_backend.connect_tcp(**kwargs)
2024-05-26 04:30:32 discollama-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     return await self._backend.connect_tcp(
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     with map_exceptions(exc_map):
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
2024-05-26 04:30:32 discollama-1  |     self.gen.throw(typ, value, traceback)
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
2024-05-26 04:30:32 discollama-1  |     raise to_exc(exc) from exc
2024-05-26 04:30:32 discollama-1  | httpcore.ConnectError: All connection attempts failed
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | The above exception was the direct cause of the following exception:
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/discord/client.py", line 441, in _run_event
2024-05-26 04:30:32 discollama-1  |     await coro(*args, **kwargs)
2024-05-26 04:30:32 discollama-1  |   File "/mnt/discollama.py", line 109, in on_message
2024-05-26 04:30:32 discollama-1  |     async for part in self.generate(content, context):
2024-05-26 04:30:32 discollama-1  |   File "/mnt/discollama.py", line 131, in generate
2024-05-26 04:30:32 discollama-1  |     async for part in await self.ollama.generate(model=self.model, prompt=content, context=context, keep_alive=-1, stream=True):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/ollama/_client.py", line 339, in inner
2024-05-26 04:30:32 discollama-1  |     async with self._client.stream(method, url, **kwargs) as r:
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
2024-05-26 04:30:32 discollama-1  |     return await anext(self.gen)
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1573, in stream
2024-05-26 04:30:32 discollama-1  |     response = await self.send(
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
2024-05-26 04:30:32 discollama-1  |     response = await self._send_handling_auth(
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
2024-05-26 04:30:32 discollama-1  |     response = await self._send_handling_redirects(
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
2024-05-26 04:30:32 discollama-1  |     response = await self._send_single_request(request)
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request
2024-05-26 04:30:32 discollama-1  |     response = await transport.handle_async_request(request)
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 365, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     with map_httpcore_exceptions():
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
2024-05-26 04:30:32 discollama-1  |     self.gen.throw(typ, value, traceback)
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
2024-05-26 04:30:32 discollama-1  |     raise mapped_exc(message) from exc
2024-05-26 04:30:32 discollama-1  | httpx.ConnectError: All connection attempts failed

And this is my ollama server running locally :

C:\Users\kilian>ollama serve
2024/05/26 04:31:21 routes.go:1008: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR:C:\\Users\\kilian\\AppData\\Local\\Programs\\Ollama\\ollama_runners OLLAMA_TMPDIR:]"
time=2024-05-26T04:31:22.002+02:00 level=INFO source=images.go:704 msg="total blobs: 24"
time=2024-05-26T04:31:22.024+02:00 level=INFO source=images.go:711 msg="total unused blobs removed: 0"
time=2024-05-26T04:31:22.026+02:00 level=INFO source=routes.go:1054 msg="Listening on 127.0.0.1:11434 (version 0.1.38)"
time=2024-05-26T04:31:22.026+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11.3 rocm_v5.7]"
time=2024-05-26T04:31:22.185+02:00 level=INFO source=types.go:71 msg="inference compute" id=GPU-XXXXXXX library=cuda compute=8.6 driver=12.4 name="NVIDIA GeForce RTX 3070 Ti" total="8.0 GiB" available="6.9 GiB"

Also when this error is happening the bot still typing on discord, you should add a condition to remove the typing when error happened.

@MeDott29
Copy link

MeDott29 commented Sep 11, 2024

OLLAMA_HOST=http://host.docker.internal:11434

worked for me.

@mxyng
Copy link
Owner

mxyng commented Sep 11, 2024

if discollama is running inside a container and ollama is not, OLLAMA_HOST must be updated to address the docker host since 127.0.0.1 refers to the container. for macos and windows, this is simply setting OLLAMA_HOST to host.docker.internal which is a magic hostname set up by docker desktop. it's not quite as easy for linux. you could try setting it to the host's internal address

@j3bx
Copy link

j3bx commented Oct 9, 2024

Workaround for linux:

  1. Open the port of the ollama server
  2. Use 0.0.0.0 as the host when starting ollama
  3. Use this IP as host in compose.yaml: ip addr show docker0 | grep -Po 'inet \K[\d.]+'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants