Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change metadata representation in DB #1572

Open
melange396 opened this issue Jan 16, 2025 · 0 comments
Open

Change metadata representation in DB #1572

melange396 opened this issue Jan 16, 2025 · 0 comments
Labels
acquisition changes acquisition logic api change affect the API and its responses enhancement mysql mysql database related python Pull requests that update Python code refactor Substantial projects to make the code do the same thing, better.

Comments

@melange396
Copy link
Collaborator

Signal metadata is stored in a large JSON-encoded text field in a DB table with only one row. This works well enough currently, but it is inefficient in a number of ways. For instance: changing any value (like updating a single signal) requires re-writing the entirety of the JSON string, and we have API methods that decode the JSON (ex 1, ex2) then perform operations on the result then re-encode it before serving it to clients. Among other things, this approach is not good for concurrency and it does not leverage the power of a DBMS.

An improvement would be to give the metadata table a schema that uses individual columns for each metric, and a row for each signal.

This can aid in making it easier to do things like #289.

@melange396 melange396 added acquisition changes acquisition logic api change affect the API and its responses enhancement mysql mysql database related python Pull requests that update Python code refactor Substantial projects to make the code do the same thing, better. labels Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
acquisition changes acquisition logic api change affect the API and its responses enhancement mysql mysql database related python Pull requests that update Python code refactor Substantial projects to make the code do the same thing, better.
Projects
None yet
Development

No branches or pull requests

1 participant