Upgrading Netbox 3.7.3 to 4.0.8 - stuck at "Installing build dependencies" - proxy issue?

[tfboy]

I'm trying to upgrade Netbox from 3.7.3 to 4.0.8 on an Ubuntu 22.04 LTS system.
The native Python is 3.8 but I have Python 3.11 now installed and that seems to work adding the PYTHON argument when launching the upgrade script.

However, it seems to hang. I get as far as:

Using cached psycopg_c-3.2.1.tar.gz
Installing build dependencies ...

And it's "stuck" there. There's the incredibly slow "rotating" /-| character but no progress. CPU usage is idle so it's not thinking.
edit: after a while, I get the following:

.
.
Collecting psycopg-c==3.2.1 (from psycopg[c,pool]==3.2.1->-r requirements.txt (line 28))
  Using cached psycopg_c-3.2.1.tar.gz (598 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error

  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [8 lines of output]
      Ignoring tomli: markers 'python_version < "3.11"' don't match your environment
      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa3281a7350>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
      WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa3281b01d0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
      WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa3281b0e50>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
      WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa3281b1ad0>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
      WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fa3281b2710>, 'Connection to pypi.org timed out. (connect timeout=15)')': /simple/setuptools/
      ERROR: Could not find a version that satisfies the requirement setuptools>=49.2.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=49.2.0
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

Now the complication to my install is that it doesn't have direct internet access, but via a proxy. In the past, I've modified the install.sh so that the proxy is piped into the pip3 commands and that's always worked fine, for example: COMMAND="pip --proxy=http://10.1.1.1:3128 install --upgrade pip"

My guess is somewhere, pypi.org is called but not with the proxy parameters and so timing out.

How can I work around this?

[dionrowney]

I had to do
yum install postgres-devel on RHEL to get psycopg_c to build
it will not install postgres database

[tfboy]

thanks, I'm on ubuntu 22.04 so the packages are a little different. I installed one, but not the right one (or the root issue is something else). I'll carry on searching.
But I do think the issue is proxy related.

edit2: OK, I think I've fixed my proxy issue, I now get past building the dependencies.
Here's where I get to:

Using cached pyyaml_env_tag-0.1-py3-none-any.whl (3.9 kB)
Using cached referencing-0.35.1-py3-none-any.whl (26 kB)
Using cached rpds_py-0.19.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (355 kB)
Using cached watchdog-4.0.1-py3-none-manylinux2014_x86_64.whl (83 kB)
Using cached text_unidecode-1.3-py2.py3-none-any.whl (78 kB)
Using cached aniso8601-9.0.1-py2.py3-none-any.whl (52 kB)
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Building wheels for collected packages: psycopg-c
  Building wheel for psycopg-c (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for psycopg-c (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/psycopg_c
      copying psycopg_c/__init__.py -> build/lib.linux-x86_64-cpython-311/psycopg_c
      copying psycopg_c/version.py -> build/lib.linux-x86_64-cpython-311/psycopg_c
      copying psycopg_c/_psycopg.pyi -> build/lib.linux-x86_64-cpython-311/psycopg_c
      copying psycopg_c/py.typed -> build/lib.linux-x86_64-cpython-311/psycopg_c
      copying psycopg_c/pq.pxd -> build/lib.linux-x86_64-cpython-311/psycopg_c
      creating build/lib.linux-x86_64-cpython-311/psycopg_c/_psycopg
      copying psycopg_c/_psycopg/oids.pxd -> build/lib.linux-x86_64-cpython-311/psycopg_c/_psycopg
      copying psycopg_c/_psycopg/endian.pxd -> build/lib.linux-x86_64-cpython-311/psycopg_c/_psycopg
      copying psycopg_c/_psycopg/__init__.pxd -> build/lib.linux-x86_64-cpython-311/psycopg_c/_psycopg
      creating build/lib.linux-x86_64-cpython-311/psycopg_c/pq
      copying psycopg_c/pq/libpq.pxd -> build/lib.linux-x86_64-cpython-311/psycopg_c/pq
      copying psycopg_c/pq/__init__.pxd -> build/lib.linux-x86_64-cpython-311/psycopg_c/pq
      running build_ext
      building 'psycopg_c._psycopg' extension
      creating build/temp.linux-x86_64-cpython-311
      creating build/temp.linux-x86_64-cpython-311/psycopg_c
      creating build/temp.linux-x86_64-cpython-311/psycopg_c/types
      x86_64-linux-gnu-gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/postgresql -I/opt/netbox-4.0.8/venv/include -I/usr/include/python3.11 -c psycopg_c/_psycopg.c -o build/temp.linux-x86_64-cpython-311/psycopg_c/_psycopg.o
      psycopg_c/_psycopg.c:42:10: fatal error: Python.h: No such file or directory
         42 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for psycopg-c
Failed to build psycopg-c
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (psycopg-c)

[candlerb]

psycopg_c/_psycopg.c:42:10: fatal error: Python.h: No such file or directory

You need to install all the dependencies as documented here:

sudo apt install -y python3 python3-pip python3-venv python3-dev build-essential libxml2-dev libxslt1-dev libffi-dev libpq-dev libssl-dev zlib1g-dev

If you had installed python3-dev then via dependencies it would have installed libpython3-dev then libpython3.XX-dev which contains the missing file.

[tfboy]

Ahh, I was missing the python3.11-dev package.
All good now :)

[pehan-dotcom]

edit2: OK, I think I've fixed my proxy issue, I now get past building the dependencies. Here's where I get to:

Hello, I'm interested how you fixed proxy issue. Did you just add "--proxy http://ip_address:port" to upgrade.sh script or something else?

I'm stuck on

Collecting django-pglocks==1.0.4 (from -r requirements.txt (line 8))
  Using cached django-pglocks-1.0.4.tar.gz (4.1 kB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fc1bcc4ac30>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fc1bca97770>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fc1bca979e0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fc1bca97bf0>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
      WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fc1bca97e00>: Failed to establish a new connection: [Errno -2] Name or service not known')': /simple/setuptools/
      ERROR: Could not find a version that satisfies the requirement setuptools>=40.8.0 (from versions: none)
      ERROR: No matching distribution found for setuptools>=40.8.0
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output

Any ideas? Thanks.

[tfboy]

Yes, that's what I used to do, but it's messy having to edit the upgrade.sh each time (and you need apply the changes to 4 different lines).
I've now set up the HTTP_PROXY and HTTPS_PROXY in my profile or the system's /etc/environment file so it will always be used by Python.
I did this not just because of the pain of modifying the upgrade script but some plugins or other scripts also rely on Internet access and it's not always easy to add the parameter in their code!

[zerihh]

a have similar problem:

i running in a Oracle Linux 8.1 and tring to install Netbox 4.1.1

Can u help me?

[pehan-dotcom]

It seems I have exactly the same problem. Also behind http proxy and stuck at psycopg dependencies. I have tried with python 3.11, 3,12 and according to official documentation here all should be in place.

If I understand it correctly, Local Installation should be done manually, but I found no exact guide all over internet. I tried everything I could.
Python, python3.12 and python-devels in place:

Compiler, libpq instaled, pg_config available in PATH :

And I'm still stuck at:

I reported this 2 months ago and till now no luck, even tried with newer netbox version 4.0.10 and 4.1.1...
Upgrading from 3.7.5 to 4.0.6

I think "Ignoring tomli: markers 'python_version < "3.11"' don't match your environment" may be the problem, but cannot solve it. It seems something points to old python version 3.9.19 even it is not default.

Any hints anyone?

[pehan-dotcom]

Ok I finally solve the problem. It seems --proxy option is not passed correctly to all modules or whatever. The final solution was bypass proxy and give the server direct connection to internet through corporate firewall (PAT). Then just run upgrade script withou ANY --proxy settings and voila:

Upgrade complete! Don't forget to restart the NetBox services:

[zerihh]

I can solve the problem by adding a variable PIP_PROXY="proxyserver:port" at the top of the upgrade.sh file. Apparently, that forces pip to always use the proxy.