Merge pull request #4888 from kobotoolbox/django-kobocat-app

Make Kobocat a Django app inside KPI

Author: noliveleger

.github/workflows/pytest.yml

@@ -19,9 +19,10 @@ jobs:
       SERVICE_ACCOUNT_BACKEND_URL: redis://localhost:6379/4
       CACHE_URL: redis://localhost:6379/3
       ENKETO_REDIS_MAIN_URL: redis://localhost:6379/0
+      KOBOCAT_MEDIA_ROOT: /tmp/test_media
     strategy:
       matrix:
-        python-version: ['3.8', '3.10', '3.12']
+        python-version: ['3.8', '3.10']
     services:
       postgres:
         image: postgis/postgis:14-3.4

Dockerfile

@@ -29,8 +29,6 @@ ENV KPI_LOGS_DIR=/srv/logs \
     TMP_DIR=/srv/tmp \
     UWSGI_USER=kobo \
     UWSGI_GROUP=kobo \
-    SERVICES_DIR=/etc/service \
-    CELERY_PID_DIR=/var/run/celery \
     INIT_PATH=/srv/init
 
 ##########################################
@@ -40,13 +38,7 @@ ENV KPI_LOGS_DIR=/srv/logs \
 RUN mkdir -p "${NGINX_STATIC_DIR}" && \
     mkdir -p "${KPI_SRC_DIR}" && \
     mkdir -p "${KPI_NODE_PATH}" && \
-    mkdir -p "${TMP_DIR}" && \
-    mkdir -p ${CELERY_PID_DIR} && \
-    mkdir -p ${SERVICES_DIR}/uwsgi && \
-    mkdir -p ${SERVICES_DIR}/celery && \
-    mkdir -p ${SERVICES_DIR}/celery_low_priority && \
-    mkdir -p ${SERVICES_DIR}/celery_beat && \
-    mkdir -p "${INIT_PATH}"
+    mkdir -p "${TMP_DIR}"
 
 ##########################################
 # Install `apt` packages.                #
@@ -63,6 +55,7 @@ RUN apt-get -qq update && \
     echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_16.x nodistro main" \
         | tee /etc/apt/sources.list.d/nodesource.list && \
     apt-get -qq update && \
+    apt-get -qq -y install openjdk-17-jre && \
     apt-get -qq -y install --no-install-recommends \
         ffmpeg \
         gdal-bin \
@@ -76,7 +69,6 @@ RUN apt-get -qq update && \
         postgresql-client \
         procps \
         rsync \
-        runit-init \
         vim-tiny \
         wait-for-it && \
     apt-get clean && \
@@ -161,28 +153,15 @@ RUN echo "export PATH=${PATH}" >> /etc/profile && \
     echo 'source /etc/profile' >> /root/.bashrc && \
     echo 'source /etc/profile' >> /home/${UWSGI_USER}/.bashrc
 
-
-# Remove getty* services to avoid errors of absent tty at sv start-up
-RUN rm -rf /etc/runit/runsvdir/default/getty-tty*
-
-# Create symlinks for runsv services
-RUN ln -s "${KPI_SRC_DIR}/docker/run_uwsgi.bash" "${SERVICES_DIR}/uwsgi/run" && \
-    ln -s "${KPI_SRC_DIR}/docker/run_celery.bash" "${SERVICES_DIR}/celery/run" && \
-    ln -s "${KPI_SRC_DIR}/docker/run_celery_low_priority.bash" "${SERVICES_DIR}/celery_low_priority/run" && \
-    ln -s "${KPI_SRC_DIR}/docker/run_celery_beat.bash" "${SERVICES_DIR}/celery_beat/run"
-
-
 # Add/Restore `UWSGI_USER`'s permissions
 # chown of `${TMP_DIR}/.npm` is a hack needed for kobo-install-based staging deployments;
 # see internal discussion at https://chat.kobotoolbox.org/#narrow/stream/4-Kobo-Dev/topic/Unu.2C.20du.2C.20tri.2C.20kvar.20deployments/near/322075
-RUN chown -R ":${UWSGI_GROUP}" ${CELERY_PID_DIR} && \
-    chmod g+w ${CELERY_PID_DIR} && \
-    chown -R "${UWSGI_USER}:${UWSGI_GROUP}" ${KPI_SRC_DIR}/emails/ && \
+RUN chown -R "${UWSGI_USER}:${UWSGI_GROUP}" ${KPI_SRC_DIR}/emails/ && \
     chown -R "${UWSGI_USER}:${UWSGI_GROUP}" ${KPI_LOGS_DIR} && \
     chown -R "${UWSGI_USER}:${UWSGI_GROUP}" ${TMP_DIR} && \
     chown -R root:root "${TMP_DIR}/.npm"
 
 
 EXPOSE 8000
 
-CMD ["/bin/bash", "-c", "exec ${KPI_SRC_DIR}/docker/init.bash"]
+CMD ["/bin/bash", "docker/entrypoint.sh"]

README.md

@@ -6,9 +6,17 @@
 
 We're open for [contributions](./CONTRIBUTING.md)!
 
+## Important notice when upgrading from any release older than [`2.024.19`](https://github.com/kobotoolbox/kpi/releases/tag/2.024.19)
+
+Prior to release [`2.024.19`](https://github.com/kobotoolbox/kpi/releases/tag/2.024.19), this project (KPI) and [KoboCAT](https://github.com/kobotoolbox/kobocat) were two separated projects.
+KoboCAT is now part of KPI code base and its repository has been archived.
+
+KoboCAT deprecation notices will be maintained in this repository.
+[More details here](./kobo/apps/openrosa/README.md)
+
 ## Important notice when upgrading from any release older than [`2.020.18`](https://github.com/kobotoolbox/kpi/releases/tag/2.020.18)
 
-Prior to release [`2.020.18`](https://github.com/kobotoolbox/kpi/releases/tag/2.020.18), this project (KPI) and [KoBoCAT](https://github.com/kobotoolbox/kobocat) both shared a common Postgres database. They now each have their own. **If you are upgrading an existing single-database installation, you must follow [these instructions](https://community.kobotoolbox.org/t/upgrading-to-separate-databases-for-kpi-and-kobocat/7202)** to migrate the KPI tables to a new database and adjust your configuration appropriately.
+Prior to release [`2.020.18`](https://github.com/kobotoolbox/kpi/releases/tag/2.020.18), this project (KPI) and [KoboCAT](https://github.com/kobotoolbox/kobocat) both shared a common Postgres database. They now each have their own. **If you are upgrading an existing single-database installation, you must follow [these instructions](https://community.kobotoolbox.org/t/upgrading-to-separate-databases-for-kpi-and-kobocat/7202)** to migrate the KPI tables to a new database and adjust your configuration appropriately.
 
 If you do not want to upgrade at this time, please use the [`shared-database-obsolete`](https://github.com/kobotoolbox/kpi/tree/shared-database-obsolete) branch instead.
 
@@ -35,7 +43,7 @@ syntax, see the documentation at the top of
 
 ## Admin reports
 
-There are several types of data reports available to superusers. 
+There are several types of data reports available to superusers.
 * Full list of users including their details provided during signup, number of deployed projects (XForm count), number of submissions, date joined, and last login: `<kpi base url>/superuser_stats/user_report/`. File being created is a CSV, so don't download immediately to wait for server to be finished writing to the file (it will download even if incomplete).
 * Monthly aggregate figures for number of forms, deployed projects, and submissions (from kobocat): `<kc server domain>/<superuser username>/superuser_stats/`
 

dependencies/pip/dev_requirements.in

@@ -11,3 +11,8 @@ pytest
 pytest-cov
 pytest-django
 pytest-env
+
+
+# Kobocat
+httmock
+simplejson