Rework composition files

Author: arnaud-morvan

.github/workflows/ci.yml

@@ -12,13 +12,13 @@ jobs:
       - uses: actions/checkout@v4
 
       - name: build
-        run: docker compose -f docker-compose-ci.yml build check
+        run: docker compose -f docker-compose-dev.yml build check
 
       - name: check
-        run: docker compose -f docker-compose-ci.yml run --rm check
+        run: docker compose -f docker-compose-dev.yml run --rm check
 
       - name: test
-        run: docker compose -f docker-compose-ci.yml run --rm check pytest --cov=maelstro tests/
+        run: docker compose -f docker-compose-dev.yml run --rm check pytest --cov=maelstro tests/
 
   frontend-check:
     runs-on: ubuntu-22.04
@@ -62,7 +62,7 @@ jobs:
       - uses: actions/checkout@v4
 
       - name: Build
-        run: docker compose build maelstro-back maelstro-front
+        run: docker compose -f docker-compose-dev.yml build maelstro-back maelstro-front
 
       - name: Login to Docker Hub
         uses: docker/login-action@v3

.gitignore

@@ -172,3 +172,4 @@ cython_debug/
 
 # Custom for this repo
 .envs-database
+docker-compose.override.yml

README.md

@@ -3,16 +3,42 @@
 geOrchestra Maelstro is an application which helps synchronise geonetwork and geoserver instances
 
 ## Docker deploy
+
+### Solo quick start (without included geOrchestra composition)
+
+Maelstro can be used outside geOrchestra.
+
+First select the `docker-compose-solo.yml` file as current composition:
+
+```bash
+ln -s docker-compose-solo.yml docker-compose.yml
+```
+
+The start the compositon:
+
+```bash
+docker compose up
+```
+
+Open : http://127.0.0.1:8080/maelstro/
+There is no authentication to access the page but if needed it can be done with basic auth in the [nginx config](./config/nginx-solo.conf) (or another way)
+
 ### Quick start with geOrchestra
 
+First select the `docker-compose-geOrchestra.yml` file as current composition:
+
+```bash
+ln -s docker-compose-geOrchestra.yml docker-compose.yml
+```
+
 Before starting development you need to setup some geOrchestra configurations.
 
 See commands documented here: [georchestra/README.md](georchestra/README.md)
 
 Now you can run the Docker composition:
 
 ```bash
-docker compose -f docker-compose-geOrchestra.yml up -d
+docker compose up -d
 ```
 
 Then application should be available at:
@@ -25,18 +51,20 @@ With credentials:
 - testadmin:testadmin
 - tmaelstro:tmaelstro
 
-### Solo quick start
-Maelstro can be used outside geOrchestra.
+### Development
 
+For development purpose, you may need to use `docker-compose-dev.yml` instead:
+
+```bash
+rm -f docker-compose.yml
+ln -s docker-compose-dev.yml docker-compose.yml
 ```
-docker compose up 
-```
-Open : http://127.0.0.1:8080/maelstro/ 
-There is no authentication to access the page but if needed it can be done with basic auth in the [nginx config](./config/nginx-solo.conf) (or another way)
 
-### Development
+And maybe add overrides for debugging:
 
-If you want to develop, rename the file [docker-compose-dev.yml.override.sample](docker-compose-dev.yml.override.sample) to `docker-compose-dev.yml.override` to apply its configuration and mounts the sources of both apps.
+```bash
+cp docker-compose.override.sampl.yml docker-compose.override.yml
+```
 
 ## Kubernetes deploy
 
@@ -59,6 +87,7 @@ The configuration is based on a YAML file containing connection information abou
 For dev use of the platform, there is a sample config in the backend folder: [dev_config.yaml](backend/dev_config.yaml). This config is used by default in the docker compo.
 
 The file has 4 distict parts:
+
 - sources
 - destinations
 - db_logging
@@ -75,6 +104,7 @@ The file has 4 distict parts:
 #### DB logging
 
 The section db_logging contains all connection information to reach a writable postgres DB to use for writing and reading operation logs:
+
 - host (default: database)
 - port (default: 5432)
 - login (default: georchestra)
@@ -90,13 +120,16 @@ Substitution of credentials (login and password) can be done for the DB configur
 The `transformations` section conatains a list of xsl transformations which can be applied to the xml metadata of source or destination servers.
 
 Each named `transformation` item must conatain
+
 - `xslt_path`: local path on the server to an xsl file in which the trasformation is defind (typically in the datadir)
 - `description`: Details of the transformation content to easily identify which transformation is applied
 
 For each geonetwork item of the source and destination servers, a specific key is added to the configuration file:
+
 - transformations: list of xslt keys to be applied to the metadata of the corresponding geonetwork server
 
 If a list of transformations is defined for both the source and destination server, the copy operation is executed in the way described below:
+
 - read metadata from source
 - apply all source transformations
 - apply all destination transformations
@@ -116,7 +149,6 @@ The logics for credentials is by decreasing order of importance:
 4. Then constant login/password keys are read
 5. If still either "login" or "password" is not defined, the credentials are considered invalid and anonymous acces is used for the instance without authentication
 
-
 #### Example
 
 (see [doc_sample_config.yaml](backend/tests/doc_sample_config.yaml)):

docker-compose-ci.yml

@@ -0,0 +1,42 @@
+# This composition add minimal development overrides over geOrchestra composition
+
+include:
+  - georchestra/geor-compose.yml
+  - georchestra/geor-compose.override.yml
+
+services:
+  maelstro-front:
+    extends:
+      file: docker-compose-geOrchestra.yml
+      service: maelstro-front
+    build:
+      context: ./frontend
+
+  maelstro-back:
+    extends:
+      file: docker-compose-geOrchestra.yml
+      service: maelstro-back
+    build:
+      context: ./backend
+      target: server
+    volumes:
+      - ./backend:/app
+      - ./config/:/etc/maelstro/:ro
+    environment:
+      MAELSTRO_CONFIG: /app/dev_config.yaml
+      DEBUG: true
+    ports:
+      - 8000:8000 # Used for frontend development
+
+  check:
+    profiles:
+      - check
+    build:
+      context: ./backend
+      target: check
+    environment:
+      MAELSTRO_CONFIG: /app/tests/test_config.yaml
+      LOCAL_LOGIN: testadmin
+      LOCAL_PASSWORD: testadmin
+    volumes:
+      - ./backend:/app

docker-compose-dev.yml

@@ -1,3 +1,5 @@
+# This is an example of production ready composition for geOrchestra
+
 include:
   - georchestra/geor-compose.yml
   - georchestra/geor-compose.override.yml

docker-compose-dev.yml.override.sample

@@ -1,6 +1,7 @@
+# This is an example of production ready standalone composition (without included geOrchestra)
 
 volumes:
-    postgresql_data:
+  postgresql_data:
 
 services:
   database:

docker-compose-geOrchestra.yml

@@ -0,0 +1,12 @@
+# Here are example of some overrides for debugging
+
+services:
+  malestro-back:
+    {}
+    # For debugging with VS Code
+    # command: >
+    #   python -m debugpy --listen 0.0.0.0:5678
+    #   /usr/local/bin/fastapi dev --host 0.0.0.0 --port 8000 --reload
+    #   maelstro/main.py
+    # ports:
+    #   - 5678:5678