feat: Feature/poem migration start | NPG-0000 #538

stevenj · 2023-08-28T06:47:32Z

Description

This is the start of the integration of Poem as the API server.
It is done in a way to allow the existing and new API server to co-exists on different ports.
This can be maintained until all endpoints have been ported.

This is general code architecture work.

It also includes a start of a local monitoring stack, which will be improved upon in a future PR.

…Poem

…nt service. Allows us to gradually migrate the API. In production we could steer the API calls in the ingress, if necessary.

…the API.

…capture.

… root.

Mr-Leshiy

Am I correct that we will move all our docker-compose files under the containers/dev-local dir and so these docker-compose files should be used also for testing purposes ?

services/voting-node/poetry.toml

src/cat-data-service/Cargo.toml

src/cat-data-service/src/service/api.rs

src/cat-data-service/src/service/utilities/metrics_tracing.rs

…t-hk/catalyst-core into feature/poem-migration-start

…onses.

…rize registration errors due to invalid signatures.

…m returns 404

…to the axum processing.

…to trace endpoints.

…d to be defined once.

containers/dev-local/docker-compose.yml

src/cat-data-service/src/axum_service/mod.rs

src/cat-data-service/src/service/generic/mod.rs

src/cat-data-service/src/service/api/health/started_get.rs

src/cat-data-service/src/service/api/mod.rs

src/cat-data-service/src/service/generic/responses/resp_5xx.rs

src/cat-data-service/src/service/utilities/middleware/tracing_mw.rs

src/cat-data-service/src/settings.rs

…tly.

stevenj · 2023-09-07T16:27:40Z

The last code I pushed got both the logging and prometheus metrics working properly and properly tagged.
There are more metrics I could add, and some I would like but are non trivial like the request and response sizes.
But the current set are completely usable and probably sufficient for 90+% of all use cases.

… fix panic handler to properly log.

…t-hk/catalyst-core into feature/poem-migration-start

) # Description - Added `registration/voter` endpoint implementation for poem service - Made small refactoring ## Type of change Please delete options that are not relevant. - [ ] Bug fix (non-breaking change which fixes an issue) - [X] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update ## How Has This Been Tested? Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration - [x] voter_test --------- Co-authored-by: Steven Johnson <[email protected]>

# Description Allow `cat-data-service` to run without connection to DB, logging errors and having a configurable Retry-After header of a 503 response code. All endpoints attempt to get a connection if none is available. ## Type of change Please delete options that are not relevant. - [ ] Bug fix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] This change requires a documentation update ## Checklist - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] Any dependent changes have been merged and published in downstream modules --------- Co-authored-by: Alex Pozhylenkov <[email protected]>

stevenj added 9 commits August 23, 2023 15:38

build(event-db): remove obsolete .env file causing warnings.

863aec8

build(cat-data-service): Update dependencies and add dependencies on …

6f73f0f

…Poem

feat(cat-data-service): Run a Poem service in parallel with the curre…

c58c88f

…nt service. Allows us to gradually migrate the API. In production we could steer the API calls in the ingress, if necessary.

feat(cat-data-service): Add stoplight as the default docs viewer for …

8d8005f

…the API.

feat(cat-data-service): Add prometheus metrics to poem service

9643163

Merge branch 'main' into feature/poem-migration-start

416dbc1

feat(local environment): Add basic monitoring stack with working log …

5da3cc9

…capture.

feat(cat-data-service): refactor and cleanup the poem api integration

ebb5644

chore(local environment): Properly exclude local from only the repo…

a411a78

… root.

stevenj changed the title ~~Feature/poem migration start~~ Feature/poem migration start | NPG-0000 Aug 28, 2023

stevenj changed the title ~~Feature/poem migration start | NPG-0000~~ feat: Feature/poem migration start | NPG-0000 Aug 28, 2023

fix(cicd): Fix opentelemetry version to 0.19 to prevent version issues

96a8a10

Mr-Leshiy reviewed Aug 28, 2023

View reviewed changes

Mr-Leshiy added 2 commits August 28, 2023 11:48

fix fmt

7e7cb61

refactor, fix issues

e3197aa

stevenj marked this pull request as draft August 28, 2023 13:15

stevenj added 7 commits August 28, 2023 22:57

feat(cat-data-service): wip code cleanups

0c24f52

Merge branch 'feature/poem-migration-start' of github.com:input-outpu…

0b6a018

…t-hk/catalyst-core into feature/poem-migration-start

feat(cat-data-service): Further cleanup for poem integration

493be6a

feat(cat-data-service): Further cleanup for poem integration

edf8b38

Merge branch 'main' into feature/poem-migration-start

ba21941

feat(cat-data-service): More wip initial poem integration work

643ac1b

docs(cat-data-service): Add documentation for all currently used resp…

5dd41f5

…onses.

minikin added the draft Draft label Sep 1, 2023

stevenj added 5 commits September 1, 2023 17:05

feat(snapshot): Add analysis to snapshot check to look for and catego…

c717d8e

…rize registration errors due to invalid signatures.

feat(cat-data-service): Add API env vars to set the hostnames

ccfe052

feat(cat-data-service): Move the panic handling to its own module

799912a

feat(cat-data-service): Add full documentation for the API top level

5a9c9bf

feat(cat-data-service): Add health endpoints and document them.

6ee6a66

stevenj added enhancement New feature or request review me PR is ready for review and removed draft Draft labels Sep 4, 2023

stevenj requested review from saibatizoku, FelipeRosa, minikin and cong-or September 4, 2023 03:36

stevenj added 4 commits September 4, 2023 14:31

feat(cat-data-service): Refactor ready to call axum directly when poe…

3b60030

…m returns 404

feat(cat-data-service): Internally proxy all unimplemented endpoints …

8c0e9be

…to the axum processing.

feat(cat-data-service): Fix prometheus on upgrade and use middleware …

65dee5f

…to trace endpoints.

refactor(cat-data-service): Cleanup API Returns so that they only nee…

a0b7e42

…d to be defined once.

stevenj requested a review from Mr-Leshiy September 5, 2023 06:32

minikin requested changes Sep 5, 2023

View reviewed changes

containers/dev-local/docker-compose.yml Show resolved Hide resolved

src/cat-data-service/src/axum_service/mod.rs Outdated Show resolved Hide resolved

src/cat-data-service/src/service/generic/mod.rs Outdated Show resolved Hide resolved