Created a guide on how to setup monitoring with grafana

[Kraego]

Hello I've created a guide how to setup grafana to monitor the inverter data, with a secured mqtt connection (TLS certs and basic auth).

See: https://github.com/Kraego/OpenDTU-Grafana-Howto/blob/main/OpenDTUGrafana.md

[tbnobody]

Thank you! I will link it within the README.md!

Did you also had a look at #317 ?
How much effort would it be do adopt your setup to the new structure? It should at least be easier to transfer the values using telegraf. But maybe there is a migration of existing logged data required.

[Kraego]

Your correct. The adaption regarding the mqtt layout (topics, subtopics, naming (serial vs. name) on the telegraf side should be easy, it just regards the topics config in [[inputs.mqtt_consumer]] section.

And yes the existing data in the telegraf buckets must then be transformed into the new layout.
If I had to do this:

• I would write queries which transform the data and store it in new buckets

from(bucket: "telegram/actual")
  |> range(start: [How`far back?])
  |> filter(fn: (r) => [Insert Transform Magic here]")
  |> to(bucket:"telegramNew/actual", org:"[theOrg]")

• Adapt retention policy on telegraf/forever - adapt downsample tasks, correct grafana query

The data migration could be tricky.

Hope that helps.

[SR7673]

Excellent. It was my plan to use OpenDTU together with a Raspi and Grafana to run a MQTT broker, archive data and display graphs.
With OpenDTU, the production can be displayed. Then, if you you use an ESP8266 to read out the IR diode of your Stromzähler, then you can also display the load of the house. The house load is PV production + mains power (where mains power can be positive or negative).
This way you can determine the ratio between self consumed energy / total produced PV energy, for each day. I like to see that into one graph, power versus daytime.
If that ratio is bad (meaning that you put most of your energy into the mains), then it may be worth to even go a step further and use a battery and second HM inverter for a night-supply of the house, and for temporary heavy loads during the day.
An extended great project! All this needs an MQTT broker which can run on a Raspi. There, the second HM could be steered by remote commands, something like in the projects of https://www.youtube.com/@VoltAmpereLux
Let's see how far that goes. But MQTT and Grafana is on my list so i will follow your guide!

[Kraego]

Sounds great, I‘m also planning to extend my monitoring to consumption. At first my plan was also to do that over the 1kW impulse, but I found a more reliable solution for my kaifa smartmeter: https://github.com/Tobias0110/EVN_Kaifa_ESP_MQTT/blob/main/README.md, maybe your meter also has modbus or something.

[SR7673]

Auf dieser Seite hier gibt es eine Rubrik "discussion", auf https://github.com/Kraego/OpenDTU-Grafana-Howto/blob/main/OpenDTUGrafana.md
gibt es die nicht. Ich hätte da mal nämlich eine Frage und die würde ich dann gern direkt auf der anderen Seite stellen...

[SR7673]

oops, now i switched to German... How could this happen!

[Kraego]

Discussions is now enabled …

[imp1sh]

Why are so many different components needed like telegraf and influxdb and mosquitto? There is an API endpoint for Promethes. Can't I directly scrape that from Prometheus?

[Kraego]

@imp1sh I think it depends on your use case. Yes you're right prometheus also persists the data in a time series database similar to influx.

My setup runs on a hosted server (accessible over the internet) so the data is published via mqtt to the broker on this server. If I had Prometheus running on this server, I would have to open a port for prometheus to scrape the data from my open-dtu (I don't like to open inbound ports to my local LAN).

If my grafana would be local, I think i would also scrape the data directly to a local prometheus, I would also skip the whole certificate stuff.

[imp1sh]

Thanks very much for the response. I already found a way and it was much easier than I thought it was.
#1230

[bovcan]

Did anyone found any tutiorial for open dtu, grafana, in Home Assistant?(rPi 4)
How do you guys install all this stuff? I tried on rpi4, there I had first issues with python, that rustc was to old version.
Than I tried Ubuntu server, there again python only 3.10 version.
The Last one I DL HA.vhdx for hypervm, but here CLI is from HA. I can see stats from OpenDTu but intergrating InfluxDB, Grafana is giving me a headache.

[Kraego]

FYI: There is also a docker stack for the whole thing which simplifies the setup: https://github.com/Kraego/OpenDTU-Grafana-Stack

[bovcan]

ok, finally got something together. running HA on Hyper VM.
This helped me:
https://github.com/markusdd/solar_dashboard/blob/main/README.md
https://www.youtube.com/watch?v=KM6UC4tMVYo