urlwatch in a python3 virtual environment failing after upgrade to Trixie

[juasiepo]

After upgrading to Trixie I have started receiving this messages:

Traceback (most recent call last):
  File "/usr/lib/python3.13/importlib/metadata/__init__.py", line 407, in from_name
    return next(iter(cls.discover(name=name)))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/juan/.urlwatch/.venv/bin/urlwatch", line 33, in <module>
    sys.exit(load_entry_point('urlwatch==2.28', 'console_scripts', 'urlwatch')())
             ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/juan/.urlwatch/.venv/bin/urlwatch", line 22, in importlib_load_entry_point
    for entry_point in distribution(dist_name).entry_points
                       ~~~~~~~~~~~~^^^^^^^^^^^
  File "/usr/lib/python3.13/importlib/metadata/__init__.py", line 960, in distribution
    return Distribution.from_name(distribution_name)
           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/importlib/metadata/__init__.py", line 409, in from_name
    raise PackageNotFoundError(name)
importlib.metadata.PackageNotFoundError: No package metadata was found for urlwatch

I deleted the virtual environment and started from scratch following the instructions on https://urlwatch.readthedocs.io/en/latest/dependencies.html

Now I am getting error regarding two missing modules that are installed:

---------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 120, in process
    data = FilterBase.process(filter_kind, subfilter, self, data)
  File "/usr/lib/python3/dist-packages/urlwatch/filters.py", line 188, in process
    return filtercls(state.job, state).filter(data, subfilter)                                                                                                                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/usr/lib/python3/dist-packages/urlwatch/filters.py", line 396, in filter
raise ImportError('Please install pdftotext')
ImportError: Please install pdftotext
---------------------------------------------------------------------------

---------------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 113, in process
data = self.job.retrieve(self)
File "/usr/lib/python3/dist-packages/urlwatch/jobs.py", line 442, in retrieve
from playwright.sync_api import sync_playwright
ModuleNotFoundError: No module named 'playwright'
---------------------------------------------------------------------------

pip install playwright pdftotext
Requirement already satisfied: playwright in ./.venv/lib/python3.13/site-packages (1.54.0)
Requirement already satisfied: pdftotext in ./.venv/lib/python3.13/site-packages (3.0.0)
Requirement already satisfied: pyee<14,>=13 in ./.venv/lib/python3.13/site-packages (from playwright) (13.0.0)
Requirement already satisfied: greenlet<4.0.0,>=3.1.1 in ./.venv/lib/python3.13/site-packages (from playwright) (3.2.4)
Requirement already satisfied: typing-extensions in ./.venv/lib/python3.13/site-packages (from pyee<14,>=13->playwright) (4.14.1)

poppler-utils are installed too

[thp]

Have you tried re-creating the venv? Does it work with the Debian package? https://packages.debian.org/trixie/urlwatch

[juasiepo]

I re-created the venv
Same error with Debian shipped urlwatch

---------------------------------------------------------------------------                                                                                                                 
Traceback (most recent call last):                                                                                                                                                          
  File "/usr/lib/python3/dist-packages/urlwatch/handler.py", line 120, in process                                                                                                           
    data = FilterBase.process(filter_kind, subfilter, self, data)                                                                                                                           
  File "/usr/lib/python3/dist-packages/urlwatch/filters.py", line 188, in process                                                                                                           
    return filtercls(state.job, state).filter(data, subfilter)                                                                                                                              
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^                                                                                                                              
  File "/usr/lib/python3/dist-packages/urlwatch/filters.py", line 396, in filter                                                                                                            
    raise ImportError('Please install pdftotext')                                                                                                                                           
ImportError: Please install pdftotext                                                         
---------------------------------------------------------------------------

[thp]

What happens if you type the following into an interactive Python3.13 prompt? Please try with both the system-installed version and the venv version (in both cases, make sure you have run "pip install pdftotext" first):

>>> import pdftotext

It could be that your /usr/lib/python3 and/or pip command-line tools refer to different Python 3 versions (e.g. 3.13 in Trixie, 3.11 in Bookworm) and that's what's messing things up. In that case, using pip3.13 when installing could help. For this, the output of the following command might clue you in on which versions are installed:

dpkg -l | grep '^ii.*python3'

[juasiepo]

@thp import pdftotext returns no error

(.venv) juan@XXXXX:~/.urlwatch$ python
Python 3.13.5 (main, Jun 25 2025, 18:55:22) [GCC 14.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pdftotext
>>>

I have run the following command to make things more clear
dpkg -l | grep '^ii.*python3'|grep "3.13"

ii  idle-python3.13                                             3.13.5-2                                  all          IDE for Python (v3.13) using Tkinter
ii  libpython3-dev:amd64                                        3.13.5-1                                  amd64        header files and a static library for Python (default)
ii  libpython3-stdlib:amd64                                     3.13.5-1                                  amd64        interactive high-level object-oriented language (default python3 version)
ii  libpython3.13:amd64                                         3.13.5-2                                  amd64        Shared Python runtime library (version 3.13)
ii  libpython3.13-dev:amd64                                     3.13.5-2                                  amd64        Header files and a static library for Python (v3.13)
ii  libpython3.13-minimal:amd64                                 3.13.5-2                                  amd64        Minimal subset of the Python language (version 3.13)
ii  libpython3.13-stdlib:amd64                                  3.13.5-2                                  amd64        Interactive high-level object-oriented language (standard library, version 3.13)
ii  libpython3.13-testsuite                                     3.13.5-2                                  all          Testsuite for the Python standard library (v3.13)
ii  python-is-python3                                           3.13.3-1                                  all          symlinks /usr/bin/python to python3
ii  python3                                                     3.13.5-1                                  amd64        interactive high-level object-oriented language (default python3 version)
ii  python3-dev                                                 3.13.5-1                                  amd64        header files and a static library for Python (default)
ii  python3-doc                                                 3.13.5-1                                  all          documentation for the high-level object-oriented language Python 3
ii  python3-examples                                            3.13.5-1                                  all          examples for the Python language (default version)
ii  python3-full                                                3.13.5-1                                  amd64        Default Python Interpreter with complete class library
ii  python3-gdbm:amd64                                          3.13.5-1                                  amd64        GNU dbm database support for Python 3.x
ii  python3-minimal                                             3.13.5-1                                  amd64        minimal subset of the Python language (default python3 version)
ii  python3-tk:amd64                                            3.13.5-1                                  amd64        Tkinter - Writing Tk applications with Python 3.x
ii  python3-venv                                                3.13.5-1                                  amd64        venv module for python3 (default python3 version)
ii  python3.13                                                  3.13.5-2                                  amd64        Interactive high-level object-oriented language (version 3.13)
ii  python3.13-dev                                              3.13.5-2                                  amd64        Header files and a static library for Python (v3.13)
ii  python3.13-doc                                              3.13.5-2                                  all          Documentation for the high-level object-oriented language Python (v3.13)
ii  python3.13-examples                                         3.13.5-2                                  all          Examples for the Python language (v3.13)
ii  python3.13-full                                             3.13.5-2                                  amd64        Python Interpreter with complete class library (version 3.13)
ii  python3.13-gdbm                                             3.13.5-2                                  amd64        GNU dbm database support for Python (v3.13)
ii  python3.13-minimal                                          3.13.5-2                                  amd64        Minimal subset of the Python language (version 3.13)
ii  python3.13-tk                                               3.13.5-2                                  amd64        Tkinter - Writing Tk applications with Python (v3.13)
ii  python3.13-venv                                             3.13.5-2                                  amd64        Interactive high-level object-oriented language (pyvenv binary, version 3.13)

Thanks for your time :)

[thp]

This is weird, the error message is here:

        if pdftotext is None:                                                                      
            raise ImportError('Please install pdftotext')         

This is set here:

try:
    import pdftotext
except ImportError:
    pdftotext = None

You can try putting some logging into except ImportError or just re-raise it like this:

try:
    import pdftotext
except ImportError:
    pdftotext = None
    raise