Monthly report 28/06/2021

[bpetit]

👋🏽 This is a first try on publishing monthly reports that should be a bit more handy than the chat for those who want to understand the directions took with this project, the roadmap and what's going on.

I'll try to publish a MR at the end of every month, but it could be a bit delayed if I'm on vacation or working for a client (if you want to help and compose those reports with me to keep the pace, please tell me ! 🙏🏽 )

Please feel free to answer to that thread to ask questions about what's in the report, suggest something, etc. That's the whole point ! 🤓

On-going work

feat: containers labels 🐳

#84 about attaching labels specific to containers to process-related metrics (identifying processes running in containers and giving relevant information about them).

This is made for docker-compose stacks and kubernetes clusters to be able to easily filter power consumption metrics of stacks/services/apps/...

The attached PR is #109 and it already involves some changes :

• prometheus async runtime is now tokio/hyper instead of actix (this is because most async dependant crates are on tokio, we try to not mix runtimes).
• we attached the newly created docker-sync crate to dependencies, this is a fork of rs-docker that has been reduced to read-only and only synchronous features. Why that ? to avoid forcing all exporters (especially simple ones) to need an async runtime to get data about containers.

Improving JSON exporter and adding timestamps to Metric struct ⌚

Issue #75, PRs #97 and #113.

This should make JSON exporter more suited to data analysis and post-processing.

The second point is to make getting timestamp about the generation of each metric easier (as close as possible to the exact moment when we get the information) in exporters. Then the timestamp could be attached to the metric as a label for example.

Fixing RAPL domains order 🔢

#114 is a fix for the order in which the RAPL domains where collected. This was pretty static and could not work on some machines. This fix should be merged soon.

Datadog exporter 🐶

#86 is a PR for a new exporter to send metrics to Datadog saas monitoring platform. This should be merged after the 3 topics above have been completed, to integrate the improvements on MetricGenerator and timestamps.

Add an option to display more than 5 processes in stdout exporter

#98 fixes #91 and should be merged soon after the above topics are completed.

Discussions to have

There is a discussion to have about MetricGenerator struct and helpers. Today it only takes a reference to Topology to get metrics in a immutable way. I think we should allow to wrap Topology owned struct in MetricGenerator to simplify the arguments we pass to exporters runtimes or loops. It would also be great to have only one MetricGenerator instance created and kept by the exporter struct. for #84 and #109 for example it would allow to drastically reduce calls to the docker socket.