Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

~1x per hour the diagnostics complains with "No data since last update." #474

Open
Rayman opened this issue Aug 31, 2022 · 2 comments
Open
Labels

Comments

@Rayman
Copy link

Rayman commented Aug 31, 2022

Please complete the following information:

  • OS and Version: Ubuntu 20.04.5 LTS
  • ROS Version: noetic
  • Built from Source or Downloaded from Official Repository: both
  • Version: 1.7.0 89faa69

Describe the bug
Randomly, approximatly 1x per hour the velodyne driver gives out a single WARN diagnostics message. The point cloud stream is still good. I have a feeling this is a threading/locking issue with the diagnostics updater.

To Reproduce

  1. Run the driver
  2. Record a bagfile for 1 hour or more
  3. Run the following command to figure out if the bug happens rostopic echo -b file.bag /diagnostics/status[0]/message | grep '\w'
  4. In plotjuggler, you can plot Events in window to visually see when the bug occurs.

Expected behavior
The point cloud data stream is good, so I expect the diagnostics to always send out OK.

Additional context
Diagnostics visualized in plotjuggler. I wrote an external diagnostics_updater that listends to the pointcloud data and it never reports a problem. Both diagnostics are visualized below.
image

This is the full diagnostics message that is produced:

level: 1
name: "velodyne_nodelet_manager_driver: velodyne_packets topic status"
message: "No data since last update."
hardware_id: "Velodyne VLP-16"
values: 
  - 
    key: "Events in window"
    value: "90"
  - 
    key: "Events since startup"
    value: "35120"
  - 
    key: "Duration of window (s)"
    value: "9.077190"
  - 
    key: "Actual frequency (Hz)"
    value: "9.914963"
  - 
    key: "Target frequency (Hz)"
    value: "9.921053"
  - 
    key: "Minimum acceptable frequency (Hz)"
    value: "4.960526"
  - 
    key: "Maximum acceptable frequency (Hz)"
    value: "14.881579"
  - 
    key: "Earliest timestamp delay"
    value: "No data"
  - 
    key: "Latest timestamp delay"
    value: "No data"
  - 
    key: "Earliest acceptable timestamp delay"
    value: "-1.000000"
  - 
    key: "Latest acceptable timestamp delay"
    value: "5.000000"
  - 
    key: "Late diagnostic update count"
    value: "0"
  - 
    key: "Early diagnostic update count"
    value: "0"
  - 
    key: "Zero seen diagnostic update count"
    value: "0"
---
@Rayman Rayman added the bug label Aug 31, 2022
@JWhitleyWork
Copy link
Contributor

Sorry it took me so long to get to this. If this issue persists, please try to verify if the drop in data are due to the data stream from the sensor being interrupted (e.g. record a pcap file during a drop-out) or from the driver. Having information about system memory utilization would help too as this could be a memory leak.

@Rayman
Copy link
Author

Rayman commented Jun 1, 2023

The strange thing is that the pointcloud data is still arriving from the ROS driver at the expected rate. The problem is purely in the diagnostics message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants