99-100% timeout errors - new install

[medic29]

Checklist

• I made sure that there are no existing issues or discussions - open issues,closed issues,open discussion, closed discussion - which may answer my question or that I could contribute my information to.
• I have read the Known Issues Wiki Page and my question / issue is not listed.
• I have provided the version of the software and the operating system in my post.
• If I have an issue with genmon not communicating with the controller, I have read and followed the serial troubleshooting guide.
• I have included the generator make and model in my submission.

Version

V1.19.05

Have you submitted your log files from the About page? Note that outbound email must be working for logs to be submitted.

I have not submitted logs

What is/are your question(s)?

Generac 15KW Home Back-up Generator fueled by Nat Gas
Evolution 1 controller (I think)
RPi 3B+ powered directly from the battery using 12v - 5v USB adapter

A little backstory, over a year ago I had setup Genmon using RPi 3b+ and connected it to my generator using the WiFi connection. I had the RPi located within the generator housing, this caused the WiFi signal to be weak, but it still connected to my network and I was able to log in to the Genmon software and control and monitor my generator. Several months ago I had noticed intermittent connection with Genmon and believed it was due to the weak WiFi signal, when I would scan the network many times it would not show up with the known IP address or MAC address. A little over a week ago I decided to run a CAT6 cable out to the generator so I could connect the RPi to the network via a wired connection. Initially when I connected the LAN cable to the RPi, it didn't connect to the network. This winter I had received battery warning for the genset and ended up replacing the battery. When the RPi didn't connect to the network via the LAN cable I suspected maybe the power adapter wasn't working correctly. I tested the battery and had correct voltage, but wasn't sure how to test the adapter and figured the easiest thing to do would be to power the RPi was via an AC adapter. The RPi still didn't connect. It turns out the SD card was corrupted. I loaded installed the genmon software on a new 32gb sd card using the latest RPi 64-bit Lite OS, I followed the guidelines in the wiki for updating the OS, then installed the genmon software using the genmonmaint.sh script. I then went out and connected the RPi to the generator and opened the genmon on my laptop, the software opened on the webpage, and I was receiving data for the CPU temp but nothing from the generator.

Maintenance :
Model : Generic Air Cooled
Generator Serial Number :
Controller Detected :
Nominal RPM : 3600
Rated kW : 15 kW
Nominal Frequency : 60 Hz
Fuel Type : Natural Gas
Generator Phase : Unknown

Controller Settings :
    Calibrate Volts : 0
    Nominal Line Voltage : Unknown V
    Rated Max Power : Unknown kW


Exercise :
    Exercise Time :

Service :
    Air Filter Service Due : or
    Oil and Oil Filter Service Due : or
    Spark Plug Service Due : or
    Battery Service Due : or
    Total Run Hours : 0.00 h
    Hardware Version :
    Firmware Version :

Monitor :

Generator Monitor Stats :
    Monitor Health : System Initializing. Not receiving data.
    Controller : Nexus, Air Cooled
    Run time : Generator Monitor running for 0:53:08.
    Generator Monitor Version : V1.19.05
    Update Available : No


Communication Stats :
    Packet Count : M: 1054, S: 0
    CRC Errors : 0
    CRC Percent Errors : 0.00%
    Timeout Errors : 1053
    Timeout Percent Errors : 99.91%
    Modbus Exceptions : 0
    Validation Errors : 0
    Sync Errors : 0
    Invalid Data : 0
    Discarded Bytes : 0
    Comm Restarts : 0
    Packets Per Second : 0.33
    Modbus Transport : Serial
    Serial Data Rate : 9600


Platform Stats :
    CPU Temperature : 122.90 F
    Pi Model : Raspberry Pi 3 Model B Rev 1.2
    Pi CPU Frequency Throttling : Has occurred.
    Pi ARM Frequency Cap : OK
    Pi Undervoltage : Has occurred.
    CPU Utilization : 0.27%
    OS Name : Debian GNU/Linux
    OS Version : 12 (bookworm)
    System Uptime : 2 days, 18:12:13
    Network Interface Used : eth0
    System Time : Wednesday May 14, 2025 12:54:27

I troubleshooted the serial cable as recommended in the wiki and everything checked out like it is documented like it should. I tried switching the wires around as suggested for the transmit/receive at the molex connector, then reconnected it to the generator and there was a change noticed where I was no longer getting timeout error, but was getting tons of CRC errors. I found this odd, but have no clue to the significance.

I thought maybe part of the problem was I was connected the the network via the LAN cable and hadn't disabled the WiFi on the RPi and maybe this was causing some sort of issue, so I disconnected the LAN cable and moved a network access point close the the RPi ensuring I had great WiFi signal and went through the process of running the genmon software (after I had switched the pins in the Molex connector back) and essentially had the same problem with very high timeout errors and genmon giving an error of "system initializing. not receiving data"

I have searched this site, read tons, spent a couple of days reading and troubleshooting. I've thought maybe it has something to do with the new RPi OS, I had experienced upgrading to the new OS on another RPi with custom software, causing it to fail and we had to make some adjustments to get that software to work with the new OS. Yet, in reading through the discussions here others are using the latest OS without issues. I did check the serial port as mentioned with this result:

medic29@genmon:~ $ ls -la /dev/ser*
lrwxrwxrwx 1 root root 7 May 11 18:17 /dev/serial0 -> ttyAMA0

medic29@genmon:~ $ dmesg | grep serial
[ 2.136175] 3f215040.serial: ttyS0 at MMIO 0x3f215040 (irq = 71, base_baud = 50000000) is a 16550
[ 2.140017] serial serial0: tty port ttyS0 registered
[ 2.172675] uart-pl011 3f201000.serial: there is not valid maps for state default
[ 2.173869] uart-pl011 3f201000.serial: cts_event_workaround enabled
[ 2.175089] 3f201000.serial: ttyAMA0 at MMIO 0x3f201000 (irq = 99, base_baud = 0) is a PL011 rev2
[ 9.631247] hci_uart_bcm serial0-0: supply vbat not found, using dummy regulator
[ 9.631512] hci_uart_bcm serial0-0: supply vddio not found, using dummy regulator

I believe the above is what is supposed to be there if I read everything correctly.

I have also tried re-running the genmonmaint.sh script a few times as suggested and haven't noticed any errors or issues.

I don't know if it could be the RS232 to TTL converter has gone bad so I have ordered another.

I have also ordered another 12v - 5v USB power adapter just in case the current one is bad so I can go back to using the battery for power.

I am looking at setting up the RPi using the older OS and trying to set up genmon using that OS since that is what I used before and got it work, although I'm thinking it shouldn't be an issue if others are using the latest OS.

I also think it should matter if I am connected to the RPi via LAN or WiFi.

As I stated I have done tons of troubleshooting off every page I have read and everything comes back as the documentation shows it should. The issue seems the software isn't connecting to the controller and this is causing the timeout errors. The question is what is causing the timeout errors? Could it be the RS232 to TTL converter even if the serial tests were okay? Has something happened to my controller on the genset which is causing the issue?

I am truly an amateur here when it comes to this stuff, but I can follow directions really well which typically gets me pretty far, but I am lost trying to figure out how all of this stuff talks or communicates to each other.

I would look into the logs and or post them if I know where to find them, how to look at them and/or upload them.

Any and all help would be appreciated. I will answer questions as best I can.

Additional information

No response

[medic29]

I am going through the serial troubleshooting page again and it states: check the log file /var/log/myserial.log , below are screenshots of my results

I apologize for the images, I couldn't figure out how to copy the data from the putty app I am using to ssh into the RPi.

I followed the advice: it could mean that another program is using the serial port. Check for multiple copies of the software running (it is best to always use the "bash ./startgenmon.sh restart" and the results:

Monitor :

Generator Monitor Stats :
    Monitor Health : System Initializing. Not receiving data.
    Controller : Nexus, Air Cooled
    Run time : Generator Monitor running for 0:04:37.
    Generator Monitor Version : V1.19.05
    Update Available : No


Communication Stats :
    Packet Count : M: 92, S: 0
    CRC Errors : 0
    CRC Percent Errors : 0.00%
    Timeout Errors : 91
    Timeout Percent Errors : 98.91%
    Modbus Exceptions : 0
    Validation Errors : 0
    Sync Errors : 0
    Invalid Data : 0
    Discarded Bytes : 0
    Comm Restarts : 0
    Packets Per Second : 0.33
    Modbus Transport : Serial
    Serial Data Rate : 9600


Platform Stats :
    CPU Temperature : 130.64 F
    Pi Model : Raspberry Pi 3 Model B Rev 1.2
    Pi CPU Frequency Throttling : Has occurred.
    Pi ARM Frequency Cap : OK
    Pi Undervoltage : Has occurred.
    CPU Utilization : 0.28%
    OS Name : Debian GNU/Linux
    OS Version : 12 (bookworm)
    System Uptime : 2 days, 19:57:59
    Network Interface Used : eth0
    System Time : Wednesday May 14, 2025 14:40:14

Checking the myserial.log again shows:
Resetting SerialDevice:ReadThread Error: /dev/serial0:device reports readiness to read but returned no data

Still lost and confused at why this isn't working.

[medic29]

I guess I have discovered the source of the original problem. The 12v-5v USB converter must have went bad.

I changed out the RS232 to TTL converter with a new one -- no change

I changed the power to the RS232 to TTL converter from the 5v pin to the 3.3v pin on the RPi -- no change

Then I thought because I had read something about everything having a common ground and I was powering the RPi using the AC converter, plus I noticed the my RS232 interface had a ground with nothing hooked to it . I tried powering the RPi with my original 12v-5v USB converter and it wouldn't even spin the small cooling fan for the RPi; when I powered the RPi from the AC adapter the RPi powered up fine. I figured if I could power the RPi via the 12v-5v converter I would get the common ground from the battery to the RPi, but since the USB converter wasn't working I would have to wire the ground to everything so I went and made some small jumpers and connected the negative from the battery to the RPi, the ground from the RPi to the RS232 interface, then powered the RPi using the AC adapter and viola!!!

Maintenance :
Model : Generic Air Cooled
Generator Serial Number : 0009904882
Controller Detected : Evolution, Air Cooled
Nominal RPM : 3600
Rated kW : 15 kW
Nominal Frequency : 60 Hz
Fuel Type : Natural Gas
Generator Phase : 1
Engine Displacement : 999 cc
kW Hours in last 30 days : 0.00 kWh
Fuel Consumption in last 30 days : 0.00 cubic feet
Run Hours in last 30 days : 0.00 h

Controller Settings :
    Calibrate Current 1 : 1480
    Calibrate Current 2 : 1423
    Calibrate Volts : 1014
    Nominal Line Voltage : 240 V
    Rated Max Power : 16 kW


Exercise :
    Exercise Time : Weekly Sunday 12:00 Quiet Mode On

Service :
    Service A Due : 192 hrs or 11/02/2025
    Service B Due : 392 hrs or 11/02/2027
    Battery Check Due : 12/18/2025
    Total Run Hours : 97.00 h
    Hardware Version : V1.06
    Firmware Version : V1.14

Monitor :

Generator Monitor Stats :
    Monitor Health : OK
    Controller : Evolution, Air Cooled
    Run time : Generator Monitor running for 0:15:36.
    Power log file size : 0.00 MB of 15.00 MB
    Generator Monitor Version : V1.19.05
    Update Available : No


Communication Stats :
    Packet Count : M: 30570, S: 30569
    CRC Errors : 0
    CRC Percent Errors : 0.00%
    Timeout Errors : 0
    Timeout Percent Errors : 0.00%
    Modbus Exceptions : 0
    Validation Errors : 0
    Sync Errors : 0
    Invalid Data : 0
    Discarded Bytes : 0
    Comm Restarts : 0
    Packets Per Second : 65.29
    Average Transaction Time : 0.0304 sec
    Modbus Transport : Serial
    Serial Data Rate : 9600


Platform Stats :
    CPU Temperature : 131.54 F
    Pi Model : Raspberry Pi 3 Model B Rev 1.2
    Pi CPU Frequency Throttling : Has occurred.
    Pi ARM Frequency Cap : OK
    Pi Undervoltage : Has occurred.
    CPU Utilization : 1.71%
    OS Name : Debian GNU/Linux
    OS Version : 12 (bookworm)
    System Uptime : 0:16:23
    Network Interface Used : eth0
    System Time : Wednesday May 14, 2025 15:36:08

The RPi apparently NEEDS to have a common ground with the generator controller for communication

Next less pressing question, above under Platform Stats: it says Pi CPU Frequency Throttling: Has occurred.
Pi Undervoltage: Has occurred.

What does this mean and is it an issue. Currently I have the RPi powered by a 3.3 amp USB power supply. I do have a replacement 12v-5v USB power adapter on order, but Amazon's shipping has started using snail mail so it might take a bit to get here, although it is enroute.

I feel so dumb since I spent last Saturday and Sunday working between my laptop and the generator trying to get this to work and it was something this simple and was probably the source of the problem with it working intermittently previously. Ugh.

[skipfire]

It's documented in multiple spots that there has to be a common ground (excluding things like opto-isolators).

Throttling means your power supply into the Pi isn't adequate and dips below 4.8v (roughly) at some point, saying it "Has occurred" means that it happened since the last boot (more power is frequently drawn during boot than other times). Voltage issues can shorten the life of your Pi and micro-SD card. Happening a few times isn't likely to hurt anything, but long term it isn't good. Make sure you have a good one, not one of the cheap boards. Motorcycle adapters are recommended and you can find higher amps on those and they are usually higher quality.

[medic29]

@skipfire

Thanks for the information!!! Originally there was a common ground when using the 12v USB adapter, then when trying to figure out why the pi wasn't connecting to the network is when I switched to the AC power supply and went forward and forgot about the common ground part until today. I've said many times, it is the simple things that stump us most times.

I have a new 12v USB adapter ordered. The one I was using is the one recommended in the wiki and this is what I purchased again. It was less than a year old and I expected it to last longer. I purchased it from Amazon, so it could be a fake I guess. Recommendations of something different or from somewhere different? As for my current AC adapter, it is an Anker 3 amp power supply or charger, shouldn't that be more than enough amperage for the RPi3B+, the board is an original Raspberry and not a knock off? I'm stumped on this power thing.