Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit the stored history #145

Open
bejanvaleriu opened this issue Oct 24, 2024 · 10 comments · May be fixed by #148
Open

Limit the stored history #145

bejanvaleriu opened this issue Oct 24, 2024 · 10 comments · May be fixed by #148

Comments

@bejanvaleriu
Copy link

Hi,

About a week ago, I installed Globalping on my Ubuntu VM, which also runs Zabbix Server. I've created several items that execute the following commands every 10 minutes:

system.run["globalping ping {$TARGET} -J -L 1 -F '396982+Berlin'"]

This setup runs for ICMP and MTR across five fixed locations (ASN + city), resulting in a total of 15 calls every 10 minutes. However, I’ve noticed that Globalping is storing its measurements in the /tmp directory without automatically clearing them. Over the past 10 days, this has resulted in the creation of approximately 1.3 million directories, all named in the format globalping_688491_1729766790000, each containing a measurements file.

Is there a configuration or setting that can be adjusted to automatically clear these directories after a certain period of time? Or maybe to not store them?

Thank you for your assistance.

Best regards,
Valeriu

@radulucut
Copy link
Collaborator

Those files are session history files and are created based on the process id and creation time. I assume, in your case, a new process runs the command each time, which will result in a new session history file.

@jimaek @MartinKolarik maybe we can add a flag i.e --disable-session at the cmd level or a global one that persists in the config?

@bejanvaleriu
Copy link
Author

For the time being, I’ve set up a cron job that runs every 2 hours to delete those directories. Is there a local patch or configuration change I can implement to prevent the creation of the session history files altogether?

@jimaek
Copy link
Member

jimaek commented Oct 24, 2024

Not at the moment, but I'm sure we will address it in the next release shortly.

It would probably be best to solve the issue of accumulating files for everyone. Something like a max 100 and delete everything else. I don't think anyone needs more than 10 previous values to be honest. Or maybe even use an embedded DB? What does everyone think?

@bejanvaleriu
Copy link
Author

One idea would be to not store session history files by default. Instead, a new parameter, --save-session, could be introduced to enable the storage of session history files only when needed. Alternatively, a single logging file could be used to store the history of all sessions (if needed).

@MartinKolarik
Copy link
Member

MartinKolarik commented Oct 24, 2024

@radulucut let's add automated cleanup:

  • when writing to a file, keep at most 10k (most recent) records in it
  • keep no more than 100 directories in total (most recent)
  • delete anything older (regardless of total count) than 7 days

Feel free to adjust the storage structure to make implementing these rules easier.

Additionally, we can add a global flag --no-history so that if someone doesn't want the history at all, they can disable it.

@jimaek

This comment was marked as outdated.

@MartinKolarik

This comment was marked as outdated.

@jimaek

This comment was marked as outdated.

@MartinKolarik

This comment was marked as outdated.

@jimaek

This comment was marked as outdated.

@MartinKolarik MartinKolarik changed the title Globalping saving measurements Limit the stored history Oct 24, 2024
@jimaek jimaek linked a pull request Nov 2, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants