Merge branch 'cms-dev:master' into master

.dockerignore

@@ -0,0 +1,6 @@
+.dockerignore
+Dockerfile
+docker-compose*.yml
+.vagrant/
+codecov/
+.dev/

.github/CONTRIBUTING.md

@@ -1,14 +1,28 @@
 # Contributing and pull request process
 
-To contribute, please send an email to [email protected], or ping us on gitter with what you plan to do (unless uncontroversial and/or small), so that we can agree on the best way to implement it.
-
-We appreciate small commits that do one thing, but also that, when possible, each commit doesn't break the master branch. Please use your best judgement for the size of the commit according to these guidelines. If a commit breaks master, we at least require to push together all commits until master is fixed.
-
-We also appreciate a tidy history, so after you write all your code, consider tidying up the commits to reflect what you did at the end, which is usually a simplified version of the process that you followed to reach the final state. Moreover, each commit should not have PEP 8 or pyflakes warnings (see below for how to make sure you don't introduce any).
-
-If your change involves more than one commit, please create a PR for each of them, unless for very small and obvious commits (read: fixing typos, comments, a few obvious lines), or unless some commit breaks master.
-
-During the review, please address all comments by creating one or more 'fixup' commits on top of the branch (no forced push). At the end, either you or one of the owners can squash appropriately the fixups.
+To contribute, please send an email to [email protected], or
+ping us on [Telegram](https://t.me/contestms) with what you plan to do (unless
+uncontroversial and/or small), so that we can agree on the best way to implement
+it.
+
+We appreciate small commits that do one thing, but also that, when possible,
+each commit doesn't break the master branch. Please use your best judgement for
+the size of the commit according to these guidelines. If a commit breaks master,
+we at least require to push together all commits until master is fixed.
+
+We also appreciate a tidy history, so after you write all your code, consider
+tidying up the commits to reflect what you did at the end, which is usually a
+simplified version of the process that you followed to reach the final state.
+Moreover, each commit should not have PEP 8 or pyflakes warnings (see below for
+how to make sure you don't introduce any).
+
+If your change involves more than one commit, please create a PR for each of
+them, unless for very small and obvious commits (read: fixing typos, comments, a
+few obvious lines), or unless some commit breaks master.
+
+During the review, please address all comments by creating one or more 'fixup'
+commits on top of the branch (no forced push). At the end, either you or one of
+the owners can squash appropriately the fixups.
 
 # Code style
 
@@ -17,9 +31,11 @@ For Python code, we generally follow [PEP 8](https://www.python.org/dev/peps/pep
 We get around Python flexible type system in several ways:
 * we try to avoid "magic" (e.g., generating or changing classes on the fly);
 * we are fairly verbose with naming, trying to help the reader with following the types;
-* we follow our type annotation system for method and function docstrings (planning to switch to [PEP 484](https://www.python.org/dev/peps/pep-0484/)); see later for the format.
+* we follow our type annotation system for method and function docstrings
+  (planning to switch to [PEP 484](https://www.python.org/dev/peps/pep-0484/));
+  see later for the format.
 
-We support Python 3 only, requiring at least version 3.8.
+We support Python 3 only, requiring at least version 3.9.
 
 # Docstring type annotation format
 

.github/ISSUE_TEMPLATE/Bug_report.md

@@ -9,7 +9,7 @@ about: Create a report to help us improve
 Sorry to hear you have trouble with CMS!
 
 We use GitHub's issues to track bugs and sometimes feature requests. If you're just looking for help with CMS, there are better channels:
-* the [gitter chat](https://gitter.im/cms-dev/cms) (currently where you are more likely to get support quickly);
+* the [Telegram chat](https://t.me/contestms), where you are more likely to get support quickly;
 * the mailing list <[email protected]>.
 
 If instead you have found a possible bug, please go ahead filing the issue using the following template. Feel free to remove non-relevant information.
@@ -28,13 +28,11 @@ If instead you have found a possible bug, please go ahead filing the issue using
 
 ### Logs
 
-[Please attach relevant logs of the component(s) that caused the problem; you can find them in /var/local/log/cms/ (if CMS was running
-installed) or in ./log (if it was running from the local copy).]
+[Please attach relevant logs of the component(s) that caused the problem; you can find them in /var/local/log/cms/.]
 
 ### System Information
 
 **CMS version:** [master, or a specific version]
-**Was CMS installed:** [yes or no]
-**Using a virtualenv:** [yes or no]
+**How was CMS installed:** [virtualenv, docker, ...?]
 **Postgres version:** [if relevant]
 **Distribution and version:** [if relevant]

.github/SUPPORT.md

@@ -4,21 +4,20 @@ To learn how to install and use CMS, please read the **documentation**,
 available at <https://cms.readthedocs.org/>.
 
 If you have questions or need help troubleshooting some problem,
-contact us in the **chat** at [gitter](https://gitter.im/cms-dev/cms),
+contact us in the **chat** on [Telegram](https://t.me/contestms),
 or write on the **support mailing list**
 <[email protected]>, where no registration is required
 (you can see the archives on
 [Google Groups](https://groups.google.com/forum/#!forum/contestms-support)).
 
 To help with the troubleshooting, you can upload on some online
 pastebin the relevant **log files**, that you can find in
-/var/local/log/cms/ (if CMS was running installed) or in ./log (if it
-was running from the local copy).
+`/var/local/log/cms/`.
 
 If you encountered a bug, please file an
 [issue](https://github.com/cms-dev/cms/issues) on **GitHub** following
 the instructions in the issue template.
 
 **Please don't file issues to ask for help**, we are happy to help
-on the mailing list or on gitter, and it is more likely somebody will
+on the mailing list or on Telegram, and it is more likely somebody will
 answer your query sooner.

.github/codecov.yml

@@ -0,0 +1,5 @@
+coverage:
+  status:
+    project:
+      default:
+        threshold: 1%

.github/dependabot.yml

@@ -0,0 +1,10 @@
+# Please see the documentation for all configuration options:
+# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
+
+version: 2
+updates:
+  - package-ecosystem: "pip"
+    directory: "/"
+    schedule:
+      interval: "daily"
+    open-pull-requests-limit: 30

.github/workflows/main.yml

@@ -6,80 +6,45 @@ on:
 
 jobs:
   test:
-    runs-on: ubuntu-20.04
-
-    services:
-      postgres:
-        image: postgres:12
-        env:
-          POSTGRES_PASSWORD: postgres
-        options: >-
-          --health-cmd pg_isready
-          --health-interval 10s
-          --health-timeout 5s
-          --health-retries 5
-        ports:
-          - 5432:5432
+    runs-on: ubuntu-22.04
 
     steps:
       - uses: actions/checkout@v2
         with:
           submodules: true
 
-      - uses: actions/setup-python@v2
-        with:
-          python-version: 3.8
-
-      - name: Install dependencies
+      - name: Check available cgroups
         run: |
-          sudo apt-get update -y
-          sudo apt-get install -y \
-            build-essential \
-            php7.4-cli \
-            haskell-platform \
-            fp-compiler \
-            python2.7 \
-            cppreference-doc-en-html \
-            cgroup-lite \
-            libcap-dev \
-            python-dev \
-            libpq-dev \
-            libcups2-dev \
-            libyaml-dev \
-            rustc \
-            mono-mcs
+          mount | grep cgroup
 
-      - name: Install requirements
+      - name: Build docker image
         run: |
-          # Use distutils from the standard library for 2to3 translation
-          export SETUPTOOLS_USE_DISTUTILS="stdlib"
-
-          pip install -U pip setuptools wheel
-          pip install -r requirements.txt
-          pip install -r dev-requirements.txt
+          docker compose -p cms -f docker-compose.test.yml build testcms
 
-          sudo $(which python) prerequisites.py -y --cmsuser=$USER install
-          python setup.py install
-
-      - name: Set up database
-        env:
-          PGPASSWORD: postgres
+      - name: Run tests
         run: |
-          psql --host=localhost --username=postgres --command="CREATE USER cmsuser WITH PASSWORD 'your_password_here';"
-
-          createdb --host=localhost --username=postgres --owner=cmsuser cmsdb
-          psql --host=localhost --username=postgres --dbname=cmsdb --command='ALTER SCHEMA public OWNER TO cmsuser'
-          psql --host=localhost --username=postgres --dbname=cmsdb --command='GRANT SELECT ON pg_largeobject TO cmsuser'
+          docker compose -p cms -f docker-compose.test.yml run --rm testcms
 
-          createdb --host=localhost --username=postgres --owner=cmsuser cmsdbfortesting
-          psql --host=localhost --username=postgres --dbname=cmsdbfortesting --command='ALTER SCHEMA public OWNER TO cmsuser'
-          psql --host=localhost --username=postgres --dbname=cmsdbfortesting --command='GRANT SELECT ON pg_largeobject TO cmsuser'
-
-          cp ./config/cms.conf.sample ./config/cms.conf
-          sed -e "s/\"cmsuser\": \"cmsuser\"/\"cmsuser\": \"$USER\"/" -i ./config/cms.conf
+      - name: Upload test results to Codecov
+        if: ${{ !cancelled() }}
+        uses: codecov/test-results-action@v1
+        with:
+          files: ./codecov/junit.xml
+          flags: unittests
+          token: ${{ secrets.CODECOV_TOKEN }}
 
-          cmsInitDB
+      - name: Upload unit test coverage results to Codecov
+        uses: codecov/codecov-action@v4
+        if: ${{ !cancelled() }}
+        with:
+          files: ./codecov/unittests.xml
+          flags: unittests
+          token: ${{ secrets.CODECOV_TOKEN }}
 
-      - name: Run tests
-        run: |
-          cmsRunTests -v --coverage --codecov
+      - name: Upload functional test coverage results to Codecov
+        uses: codecov/codecov-action@v4
+        if: ${{ !cancelled() }}
+        with:
+          files: ./codecov/functionaltests.xml
+          flags: functionaltests
+          token: ${{ secrets.CODECOV_TOKEN }}

.gitignore

@@ -1,4 +1,5 @@
 *~
+.DS_Store
 *.mo
 .*.sw?
 *.pyc
@@ -11,3 +12,6 @@ cms.egg-info
 cache/
 lib/
 log/
+.vagrant/
+codecov/
+.dev/

.readthedocs.yml

@@ -0,0 +1,13 @@
+# Read the Docs configuration file for Sphinx projects
+# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details
+
+# Required
+version: 2
+
+build:
+  os: ubuntu-22.04
+  tools:
+    python: "3.12"
+
+sphinx:
+  configuration: docs/conf.py

.vscode/extensions.json

@@ -0,0 +1,12 @@
+{
+    "recommendations": [
+        // Auto-format Python code with Ruff linter
+        "charliermarsh.ruff",
+        // To apply settings from .editorconfig file
+        "editorconfig.editorconfig",
+        // Python intellisense support
+        "ms-python.python",
+        // Python language server and type checker
+        "ms-python.vscode-pylance",
+    ]
+}

.vscode/settings.json

@@ -0,0 +1,23 @@
+{
+    "python.analysis.typeCheckingMode": "basic",
+    "files.insertFinalNewline": true,
+    "editor.formatOnSave": true,
+    "editor.formatOnSaveMode": "modifications",
+    "[python]": {
+        "editor.defaultFormatter": "charliermarsh.ruff",
+    },
+    "[jsonc]": {
+        "editor.defaultFormatter": "vscode.json-language-features",
+    },
+    "[javascript]": {
+        "editor.defaultFormatter": "vscode.typescript-language-features",
+    },
+    "[css]": {
+        "editor.defaultFormatter": "vscode.css-language-features",
+    },
+    "[html]": {
+        // Doesn't work properly with template files, for example with things
+        // like {{ url("page") }} becoming {{ url(" page") }}
+        "editor.formatOnSave": false,
+    },
+}

Dockerfile

@@ -0,0 +1,59 @@
+# syntax=docker/dockerfile:1
+FROM ubuntu:24.04
+
+RUN apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
+    build-essential \
+    cgroup-lite \
+    cppreference-doc-en-html \
+    fp-compiler \
+    git \
+    ghc \
+    libcap-dev \
+    libcups2-dev \
+    libffi-dev \
+    libpq-dev \
+    libyaml-dev \
+    mono-mcs \
+    openjdk-8-jdk-headless \
+    php-cli \
+    postgresql-client \
+    pypy3 \
+    python3-pip \
+    python3.12 \
+    python3.12-dev \
+    rustc \
+    shared-mime-info \
+    sudo \
+    wait-for-it \
+    zip
+
+# Create cmsuser user with sudo privileges
+RUN useradd -ms /bin/bash cmsuser && \
+    usermod -aG sudo cmsuser
+# Disable sudo password
+RUN echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
+# Set cmsuser as default user
+USER cmsuser
+
+RUN mkdir /home/cmsuser/cms
+COPY --chown=cmsuser:cmsuser requirements.txt dev-requirements.txt /home/cmsuser/cms/
+
+WORKDIR /home/cmsuser/cms
+
+RUN sudo pip3 install --break-system-packages -r requirements.txt
+RUN sudo pip3 install --break-system-packages -r dev-requirements.txt
+
+COPY --chown=cmsuser:cmsuser . /home/cmsuser/cms
+
+RUN sudo python3 setup.py install
+
+RUN sudo python3 prerequisites.py --yes --cmsuser=cmsuser install
+
+RUN sudo sed 's|/cmsuser:your_password_here@localhost:5432/cmsdb"|/postgres@testdb:5432/cmsdbfortesting"|' ./config/cms.conf.sample \
+    | sudo tee /usr/local/etc/cms-testdb.conf
+RUN sudo sed 's|/cmsuser:your_password_here@localhost:5432/cmsdb"|/postgres@devdb:5432/cmsdb"|' ./config/cms.conf.sample \
+    | sudo tee /usr/local/etc/cms-devdb.conf
+
+ENV LANG C.UTF-8
+
+CMD [""]