Skip to content

Conversation

@ivanucherdzhiev
Copy link

@ivanucherdzhiev ivanucherdzhiev commented Jan 19, 2026

fix Failed to establish a new connection: [Errno 111] Connection refused

after tracing (with strace) whats going on i saw that instead of UNIX socked used to connect to to /run/user/1001/podman/podman.sock the python library urllib3 ignores the UNIX socked and uses AF_INET and tries to connect with TCP as it is connecting to HTTP server.

DEBUG:Vanilla::Async:Running async job [<function Monitor.read at 0x7f304bf62520>].

/run/user/1000/podman/podman.sock

2026-01-19T04:53:28

DEBUG: Sending request via Unix Socket: /run/user/1000/podman/podman.sock

DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): localhost:80

[pid 30574] connect(13, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)

[pid 30574] connect(13, {sa_family=AF_UNIX, sun_path="/var/run/nscd/socket"}, 110) = -1 ENOENT (No such file or directory)

[pid 30574] sendto(13, [{nlmsg_len=20, nlmsg_type=RTM_GETADDR, nlmsg_flags=NLM_F_REQUEST|NLM_F_DUMP, nlmsg_seq=1768801932, nlmsg_pid=0}, {ifa_family=AF_UNSPEC, ...}], 20, 0, {sa_family=AF_NETLINK, nl_pid=0, nl_groups=00000000}, 12) = 20

[pid 30574] connect(13, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("127.0.0.1")}, 16) = 0

[pid 30574] connect(13, {sa_family=AF_INET6, sin6_port=htons(80), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_scope_id=0}, 28) = 0

[pid 30574] connect(13, {sa_family=AF_INET6, sin6_port=htons(80), sin6_flowinfo=htonl(0), inet_pton(AF_INET6, "::1", &sin6_addr), sin6_scope_id=0}, 28 <unfinished ...>

[pid 30565] +++ exited with 0 +++

[pid 30574] <... connect resumed>) = -1 ECONNREFUSED (Connection refused)

after some research i found out that this behavior is because of HTTPConnectionPool so i tried to override it with a custom one which forces the use of the UNIX socked

class SocketPoolManager(PoolManager):
    def __init__(self, sock_addr):
        self.sock_addr = sock_addr
        super().__init__()

    def _new_pool(self, scheme, host, port, request_context=None):
        # This forces urllib3 to use our Socket Pool instead of a network pool
        return SocketConnectionPool(self.sock_addr)

this combined with this line

self.poolmanager = SocketPoolManager(self.__sock_addr)

forces the use of the UNIX socket to connect to the /run/user/1001/podman/podman.sock

__socket_path = f"{os.environ.get('XDG_RUNTIME_DIR')}/podman/podman.sock" this change is done in order to avoid hard coding the user UUID but instead getting it from the environment variable XDG_RUNTIME_DIR

this commit adress #62

@NN708
Copy link
Member

NN708 commented Jan 20, 2026

@ivanucherdzhiev Thanks for the thorough analysis and the fix. However, this custom implementation adds some complexity and long-term maintenance overhead.

I suggest we consider using the Podman Python SDK instead of handling this logic manually. Could you explore whether podman-py would be feasible for this? If so, we could adopt a cleaner solution.

@ivanucherdzhiev
Copy link
Author

@NN708 thank you very much for your time. I believe you are absolutely right. I will explore how that might be done with podman python SDK. Definitely that would be a better approach in the long run, I believe this problem occurred because of some changes in the urllib3.

Should I cancel the PR or leave it as a temporary solution until better approach is done?

@NN708
Copy link
Member

NN708 commented Jan 20, 2026

@ivanucherdzhiev Feel free to keep the PR open for now. Once your new implementation is ready, you can update the branch with a force push. This will keep all the discussion in one place.

@NN708 NN708 linked an issue Jan 20, 2026 that may be closed by this pull request
@ivanucherdzhiev
Copy link
Author

ivanucherdzhiev commented Jan 20, 2026

@NN708 okay thank you.
I am researching and it looks like PodmanClient.events() method, which uses the podman.domain.events.EventManager might be the right tool. I will try it out in the following days and I will let you know the results

Thank you again for your time!

@ivanucherdzhiev
Copy link
Author

@NN708 I made it work with podman python SDK and i think its much cleaner now. Please check it out and let me know what do you think.

@ivanucherdzhiev
Copy link
Author

ivanucherdzhiev commented Jan 21, 2026

Sorry I made second push because I forgot to remove unneeded test stuff from the first one

The new implementation uses PodmanClient to connect to the podman socket and then the event method to check for events

since = Monitor.__last_read, this set from what time event to be searched

until = now, this sets until what time to search for events. From what I tested I found out that this need to be set or else the event read act as a stream instead of just poll

filters = Monitor.watch_events, this is the filter for which events to search for. Filters need to be set as filter=value

decode = True this changes the output in format iterator[dict[str, ANY]]

elif event["status"] == "died": that change was needed because the PodmanClient.event return event status died instead of die

Copy link
Member

@NN708 NN708 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The client.events() appears to support datetime directly for since parameter, so we should be able to just use datetime.now() and remove the __get_datetime() function.

Regarding the code's functionality, I'm not sure if it's working as intended. My tests show that it does eliminate the error, but the status in the GUI still doesn't seem to update.

@ivanucherdzhiev
Copy link
Author

ivanucherdzhiev commented Jan 21, 2026

Hmm that interesting on my tests it works perfectly maybe there is mismatch of podmansdk versions? I will make pictures

__get_datetime() I will remove it right now

@NN708 if you are testing without setting until param the code wont work correctly in needs to be set

@ivanucherdzhiev
Copy link
Author

ivanucherdzhiev commented Jan 21, 2026

@NN708 i made few small changes

  1. i removed the __get_datetime method
  2. i tested with just datetime.now() and the script hangs in events_list = list(events) - it looks like it does that when parameter until is ahead in time , the only way it works for me is when i set the time like this datetime.now(UTC).replace(tzinfo=None) could you test that in your environment too please?
  3. i added some debug logs for easier debbugging but you need to start apx-gui with logging.basicConfig(level=logging.DEBUG) in main.py in order to see that logs
  4. i handled the case where XDG_RUNTIME_DIR is not set then default is '/run/user/1000'
  5. I found out that it sometimes misses events and i believe i handled that case by setting Monitor.__last_read = now this sets the next search since param to be exactly to this ones until param
Screenshot 2026-01-21 082455 Screenshot 2026-01-21 082519 Screenshot 2026-01-21 082536

Thank you very much for your time and support!

@NN708 NN708 force-pushed the monitor-fix-clean branch from e45f735 to 48d6153 Compare January 21, 2026 07:35
@NN708
Copy link
Member

NN708 commented Jan 21, 2026

  1. i tested with just datetime.now() and the script hangs in events_list = list(events) - it looks like it does that when parameter until is ahead in time , the only way it works for me is when i set the time like this datetime.now(UTC).replace(tzinfo=None) could you test that in your environment too please?

That's right, datetime.now() does not include the timezone information, and datetime.now(UTC) works.

  1. i handled the case where XDG_RUNTIME_DIR is not set then default is '/run/user/1000'

I have also addressed the scenario where UID is not 1000.

Copy link
Member

@NN708 NN708 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am still unable to retrieve events. The response is always empty:

DEBUG:Vanilla::Async:Running async job [<function Monitor.read at 0x7ff4b6da5440>].
DEBUG:urllib3.connectionpool:Starting new HTTP connection (1): %2frun%2fuser%2f1001%2fpodman%2fpodman.sock:80
DEBUG:urllib3.connectionpool:http://%2frun%2fuser%2f1001%2fpodman%2fpodman.sock:80 "GET /v5.6.0/libpod/events?filters=%7B%22event%22%3A+%5B%22start%22%2C+%22died%22%5D%7D&since=1768980553&stream=True&until=1768980555 HTTP/1.1" 200 None
DEBUG:apx_gui.core.monitor:DEBUG: Window was empty. Try starting/stopping a container now.

Testing Podman directly indicates that this appears to be an issue with Podman itself. Running the following command returns nothing, even when an event has occurred:

$ podman events --since "1768980526" --until "1768980555" --filter "event=died" --filter "event=start"

Events are only returned when the until parameter is set to a timestamp after current time:

$ podman events --since "1768980526" --until "1769000000" --filter "event=died" --filter "event=start"
2026-01-21 15:28:57.158083386 +0800 CST container died 33de39acc5324033f067696398294793c158dd1008fd8009a564ff624cb84d0a (image=docker.io/library/alpine:latest, name=apx-test, manager=apx, name=test, nvidia=true, stack=alpine, distrobox.unshare_groups=0)

Since the root cause lies outside our implementation, I approve the PR. I would also appreciate feedback from other developers.

@NN708 NN708 requested a review from a team January 21, 2026 08:00
@ivanucherdzhiev
Copy link
Author

i upgraded the version and everything works perfectly!

Thank you again!

@NN708
Copy link
Member

NN708 commented Jan 21, 2026

@ivanucherdzhiev Thank you for your contribution! ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Requests Max retries exceeded errors in 1.0.2

2 participants