Skip to content

Feat/geom sites groups#27

Open
mvergez wants to merge 26 commits intodev-suivi-eolienfrom
feat/geom-sites-groups
Open

Feat/geom sites groups#27
mvergez wants to merge 26 commits intodev-suivi-eolienfrom
feat/geom-sites-groups

Conversation

@mvergez
Copy link

@mvergez mvergez commented Jan 3, 2023

Objectifs

Ajouter une colonne geom (SRID 4326) aux groupes de sites pour que l'utilisateur puisse tracer la géométrie. Ceci, afin que cette dernière ne soit pas définie par l'agrégation des géométries des sites qui sont liés au groupe de sites comme cela est fait aujourd'hui.

A faire

  • Ajout de la colonne geom dans le modèle TMonitoringSitesGroups
  • Ajout de la migration alembic

Question

Dans la migration, la colonne geom est définie comme pouvant être non nulle. En effet, pour que la colonne puisse être ajoutée, il est nécessaire qu'elle puisse être nulle pour satisfaire les données existantes. Mais ceci doit être temporaire le temps de remplir cette colonne. Mais comment le faire ?

  • Remplir la colonne en agrégeant les géométries des sites lié à chaque groupe. Mais que faire pour les groupes sans sites ? Les supprimer ? Prévenir l'utilisateur de les supprimer ou de les associer à des sites avant d'effectuer la migration ?
  • Laisser la possibilité à la colonne geom d'être nulle et renvoyer l'agrégation des géométries de sites comme fait aujourd’hui ?
  • Autre ?

linked to #4

@mvergez
Copy link
Author

mvergez commented Jan 3, 2023

@camillemonchicourt
Copy link
Collaborator

camillemonchicourt commented Jan 3, 2023

Voici ce qui était imaginé pour ce sujet :

  • Actuellement la géométrie d'un groupe de sites est généré à la volée, en affichant un polygone convex sur la base des géométries des sites qui lui sont associés
  • L'idée était de proposer à l'utilisateur de dessiner la géométrie du groupe de site quand il créé celui-ci mais que cela reste optionnel
  • Si l'utilisateur ne créé pas de géométrie pour le groupe de sites, alors la géométrie utilisée/affichée resterait celle du polygone convex des sites qui le composent
  • Cela pose juste un soucis éventuellement tant que le site n'a pas de site associé. Dans ce cas, on ne peut pas l'afficher sur la carte. Mais peut-être n'est-ce pas grave ? On ne l'affiche alors que dans la liste

@georgemoon georgemoon added this to the Sprint 2 (NS 257) milestone Jan 4, 2023
Maxime Vergez and others added 14 commits January 10, 2023 16:10
Merged feat/package-module-alembic
6d55b19 
commit 70b0139
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 15:22:56 2022 +0100

    style: apply black

commit 2badedf
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 15:11:41 2022 +0100

    chore(db): removed unused import

commit 57d9b52
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 14:47:48 2022 +0100

    feat(db): improved & finished downgrade func

commit 211780f
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 09:32:12 2022 +0100

    feat(db): wip: add upgrade and began downgrade

    Downgrade migration need a little bit more work due to the use of
    sql alchemy and not plain text sql

commit 0b5a3b8
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Mon Dec 5 11:40:32 2022 +0100

    feat: packaged module
Merged feat/bib-categorie-site
466a6e9 
Squashed commit of the following:

commit 2bdc22a
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Fri Dec 9 14:59:31 2022 +0100

    chore: changed json_schema into config

commit 4279389
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 18:37:41 2022 +0100

    feat(db): add foreign key from site to cat

    From t_base_sites to bib_categorie_site

commit 5cb5dfa
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 16:47:27 2022 +0100

    feat(api): add admin interface

commit 559a0ec
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 16:46:27 2022 +0100

    feat(db): add bib_categorie_site table

    Add migration and model
@andriacap
feat: [6.2] Page d'accueil module monitoring layout et config title e…
a625733 
…t description #2 (#15)

* feat: [6.2] Page d'accueil modul monitoring layout et config title et description #2

* chore(config): applied black & removed imports
@mvergez
Feat/monitoring sites (#16)
697823d 
* feat(api): wip began add site routes + tests

With site categories
Also add tests

* feat(api): add more routes

* test(api): add tests and fixtures

* style(api): applied black

* feat(db): add migration to remove id_module

Column in t_sites_groups

* refactor(api): move utils for routes from sites

* feat(api): wip: add sites groups route

* test(api): wip: begin adding fixture site_groups

* fix: remove id_module in all models

* chore: rename route for better consistency

* tests: moved site_groups in tests and add tests

* chore(api): applied black

* refactor(api): add filter params function

And refact routes to use it
@mvergez @andriacap
Feat/site type categories and module categorie (#18)
e717216 
* feat(api): add association table with alembic

Add model in backend and alembic migration

Reviewed-by: andriac
[Refs ticket]: #3

* test: WIP add test to see new relationship

Adding test to see if categories are showing up when we call module

Reviewed-by: andriacap
[Refs ticket]: #3

* feat: add type site - categorie relation

WIP - add selectfield to get type site in admin module

Reviewed-by: andriac
[Refs ticket]: #3

* feat(api): Flask admin and routes categories

Clean code for change label list and form selectfield for
the BibCategorieView in Flask Admin

Add utils routes to get all subtable relationship in order to
get back the label type site

Review-by: andriac
[Refs ticket]: #3

* refactor: remove paginate_nested

For depth in as_dict()

* test: fix tests due to as_dict depth

* style: applied black and isort

* chore: remove unused import

Co-authored-by: Andria Capai <andria_capai@natural-solutions.eu>
@mvergez
Feat/edit categories module (#19)
8b09df2 
* tests: make tests and fixtures work for modules

By changing the way a monitoring module is created in the fixture
Add two tests to check the relationship

* feat(api): add categories in edit module

* style: applied black
@mvergez
test: refactor fixtures to load them automatically (#20)
1d58043 
Need to set the current working directory of pytest the directory
where the gn_monitoring_module repo is (so with the frontend etc.)
@mvergez @andriacap
test: move test_route in parent dir (#17)
4b0139a 
clean: Merge la PR move test_route 
Co-authored-by: Andria Capai <andria_capai@natural-solutions.eu>
@mvergez @andriacap
Feat/create marshmallow schemas and remove id_module (#21)
aa379ee 
* feat(api): remove id_module from sites_complements

Co-authored-by: andriacap <andriacap@users.noreply.github.com>

* feat(api): create schema for sites_groups

* test: add sites_group to site fixture

* test: wip add test for sites_groups schemas

* chore(api): remove depth parameter from paginate

* test: updated to work with sites_group schema

* feat: categorie site with marshmallow

Test and marshmallow create/refactor to
adapt for bibcategorie site paginate

WIP : Adapt load_only site_type in test to "assert"
same object when initiate BibCategorieSite

[Refs ticket]: #3

* feat(api): route /sites/categories/id with schema

Changing the route to return a dump Marshmallow schema
BibCategorieSitesSchema

Reviewed-by: andriac
[Refs ticket]: #3

* test(api): routes get categoires label

Change the "as_dict" by schema.dump in order
to use the Marshmallow schema created

Reviewed-by: andriac
[Refs ticket]: #3

* feat(api): Sites: cols to geoserializable + schema

* style(api): applied black

* test(api): add test for Site Schema

* style(api): applied black to test_site

* refactor(api): instantiate schema once

Instead of for each all() iteration

* chore(api): remove useless comments

* chore(api): remove useless comments and imports

Co-authored-by: andriacap <andriacap@users.noreply.github.com>
Co-authored-by: Andria Capai <andria_capai@natural-solutions.eu>
@mvergez
refactor(api): remove id_type in admin (#22)
6a135d4 
To replace it with a bib nomenclature type: TYPE_SITE
@mvergez
style(config): rename attribut label of categories (#23)
ecc02a4
@mvergez
Fix/paginate utils (#24)
15edb4b 
* fix(api): add int conversion for limit/offset

* test(api): add test for get_limit_offset

* fix(api): max_per_page => per_page & test

* test(api): update test with changes on schema

Since marshmallow schemas, the json returned by pagination has
changed for items and not sites
@mvergez
Feat/improve filter (#25)
a38eda5 
* feat(api): add Query Class to sites, grps & cats

* feat(api): add sort and fix _get_model

Via _get_entity

* test(api): test sort query

* feat(api): add api sort/sort_dir params

To be able to sort through REST Api

* fix(api): check if integer to avoid using ilike

In filter_by_params

* test(api): add test to check filter integer
@mvergez
fix(config): changed categories into items (#29)
83aa91a 
* Merged feat/package-module-alembic

commit 70b0139
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 15:22:56 2022 +0100

    style: apply black

commit 2badedf
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 15:11:41 2022 +0100

    chore(db): removed unused import

commit 57d9b52
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 14:47:48 2022 +0100

    feat(db): improved & finished downgrade func

commit 211780f
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 09:32:12 2022 +0100

    feat(db): wip: add upgrade and began downgrade

    Downgrade migration need a little bit more work due to the use of
    sql alchemy and not plain text sql

commit 0b5a3b8
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Mon Dec 5 11:40:32 2022 +0100

    feat: packaged module

* Feat/edit categories module (#19)

* tests: make tests and fixtures work for modules

By changing the way a monitoring module is created in the fixture
Add two tests to check the relationship

* feat(api): add categories in edit module

* style: applied black

* fix(config): changed categories into items

Following the change with marshmallow schemas
Maxime Vergez and others added 10 commits January 10, 2023 16:16
fix: rebase failure: missing geonature
dbcf9a0
@mvergez @andriacap
Fix/db migrations (#31)
a122e9b 
* feat(db): upgrade down_revision following rebase

Since rebase with develop: changed the down_revision number

* fix(db): fix bind params enabling downgrade

Beforehand the downgrade was not possible...

* refactor(db): removed cor_site_type_category

* refactor(db): changed category into type in cor

* refactor(db): create cor_type_site

* fix(db): renamed column

* refactor(api): update models to fit migrations

* fix(db):change bib_categorie_site to bib_type_site

Adding :
cor_site_module
cor_site_type
revision alembic to create function and trigger in order to add
bib_type_site but only with nomenclature 'TYPE_SITE'
upgrade and downgrade works

[Refs ticket]: #3
Reviewed-by: andriac

* fix(api): updated models from migrations

* fix(api): wip: fix admin following migrations

* fix(api): update routes and tests

To match migration changes

* feat: flask admin bib_type_site

Change bib_categories to bib_type_site into flask admin
Adding filtering in list label_fr of type_site to secure the unique
constraint

Reviewed-by: andriac
[Refs ticket]: #3

* fix(api): updated schema to match models

* fix(api): module edition

* style(api): uniformize type_site

* style(api): change relationship name for type_site

* feat(api): validator admin

* fix(api): make unique BibTypeSite in admin

* test(api): fix test when existing nomenclatures

In database

Co-authored-by: Andria Capai <andria_capai@natural-solutions.eu>
@mvergez
perf(api): improved loading of modules (#30)
b0d968e 
* Merged feat/package-module-alembic

commit 70b0139
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 15:22:56 2022 +0100

    style: apply black

commit 2badedf
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 15:11:41 2022 +0100

    chore(db): removed unused import

commit 57d9b52
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 14:47:48 2022 +0100

    feat(db): improved & finished downgrade func

commit 211780f
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Tue Dec 6 09:32:12 2022 +0100

    feat(db): wip: add upgrade and began downgrade

    Downgrade migration need a little bit more work due to the use of
    sql alchemy and not plain text sql

commit 0b5a3b8
Author: Maxime Vergez <maxime_vergez@natural-solutions.eu>
Date:   Mon Dec 5 11:40:32 2022 +0100

    feat: packaged module

* perf(api): improved loading of modules

When there are loads of datasets, the relationship takes a lot of
time to load, and is useless for the route that uses it.

* fix: due to wrong rebase

Fix: newline on data_utils.py
Fix: relationships are not joined anymore since there is no need of
them
Chore: update comment
@mvergez
Fix/pagination (#28)
e549193 
* fix(api): page - 1 that returned the wrong page

* test(api): add test for paginate
@mvergez
style(api): restore data_utils spaces (#33)
ec1bf2a
@andriacap
Fix/db migrations checkconstrainton bib_type_site.id_nomenclature (#34)
a4c7f29 
* fix(db): change trigger to constraint (migrations)

Delete the trigger and create check constraint on id_nomenclature column

Reviewed-by: andriacap
[Refs ticket]: #3

* fix(db) : apply black on migration file

Apply black
[Refs_ticket]: #3
@mvergez
fix(api): invert filter condition with Unicode (#35)
2834de1 
So that it will fallback to == most of the time and ilike when
just Unicode
@mvergez
fix(db): add NOT VALID in constraint for bib_type_site (#36)
0e15017 
Cannot use alembic to do this because need sqlalchemy >= 1.4.32
feat(db): migration to add geom col to sites_group
20994e3
feat(api): add geom column to model
16b9f5d 
To TMonitoringSitesGroups model
@mvergez mvergez force-pushed the feat/geom-sites-groups branch from 1ccdef3 to 16b9f5d Compare January 17, 2023 09:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

Sprint 4 (NS 259)

Development

Successfully merging this pull request may close these issues.

4 participants

@mvergez @camillemonchicourt @georgemoon @andriacap