Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor 'metrics' database table and datasource_metrics worker #420

Open
sal-uva opened this issue Apr 5, 2024 · 1 comment
Open

Refactor 'metrics' database table and datasource_metrics worker #420

sal-uva opened this issue Apr 5, 2024 · 1 comment
Assignees
Labels
(mostly) back-end Primarily involves the back-end (e.g. post-processors, scheduler or crawlers).

Comments

@sal-uva
Copy link
Collaborator

sal-uva commented Apr 5, 2024

Since 4CAT will no longer be used to locally store data, the information that is now stored in the 'metrics' table (which was used to e.g. keep track of counts of posts per day) should be moved to 4cat-scrapers.

@sal-uva sal-uva added the (mostly) back-end Primarily involves the back-end (e.g. post-processors, scheduler or crawlers). label Apr 5, 2024
@sal-uva sal-uva self-assigned this Apr 5, 2024
@stijn-uva
Copy link
Member

The metrics table is also used to cache metrics relating to 4CAT's use of disk space. It's not a perfect fit for that, so the table could be refactored if it doesn't need to store actual datasource-specific metrics anymore, but it shouldn't be deleted because this functionality needs to remain inside 4CAT itself.

@stijn-uva stijn-uva changed the title Remove 'mectrics' database table and functionality Refactor 'metrics' database table and datasource_metrics worker Apr 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
(mostly) back-end Primarily involves the back-end (e.g. post-processors, scheduler or crawlers).
Projects
None yet
Development

No branches or pull requests

2 participants