Bags with malformed rosjava message definitions break some rosbag operations

[trey0]

[Oops, first opened this issue as https://github.com/nasa/isaac/issues/32 by mistake. Belongs in this repo.]

Astrobee FSW bags often contain messages produced by the guest science manager running on the HLP implemented in Android Java, publishing messages using rosjava. The relevant message topics start with /gs/gs_manager.

These bags cause certain rosbag API calls and command-line utilities, such as rosbag check and rosbag filter, to fail with the following non-intuitive error message:

genmsg.msg_loader.MsgNotFound: Cannot locate message [Header]: unknown package [std_msgs] on search path [{}]

This external issue discusses the problem: jacknlliu/development-issues#39 . It seems to be related to rosjava not including the expected dependency information along with its message definitions.

The error message makes it look suspiciously like the ROS environment is not activated or somehow misconfigured, but it occurs even when all the other messages in the FSW bag work fine, as you can verify by filtering out the /gs/* messages and making the same rosbag calls. (Also, rosmsg info std_msgs/Header works fine.)

We've observed this with FSW bags, making rosbag calls in the standard environment on the astrobeast server. An example bag is https://hivemind.ndc.nasa.gov/freeflyer/2022-01-03_100/robot_data/SN003/bags/20220103_1237_ars_default.bag

It's probably not within our project scope to fix this rosjava problem, but there should be some documented guidance on how to cope with it.

[marinagmoreira]

I've tested this tool: https://github.com/gavanderhoorn/rosbag_fixer and it works nicely

python fix_bag_msg_def.py --use-local-defs 20220103_1237_ars_default.bag out.bag

[trey0]

I've tested this tool: https://github.com/gavanderhoorn/rosbag_fixer and it works nicely

Hm. I tested a similar tool before that didn't work, or maybe I called it the wrong way, but I agree your way works fine on astrobeast. Nice find!

With this new information, I would recommend the following steps to close this issue:

• I think there should be an entry in the FAQ so you can search on the weird error message and get Marina's answer about how to fix the bag. I would be happy to write that, but I don't know where the FAQ is :)
• Quite possibly it should become part of the Facility team's standard procedure to fix the bag after downlink, so most users never have to deal with a broken bag. It's kind of a no-brainer good idea if there is space on hivemind to keep both the original unfixed bag and the fixed bag. But if keeping both copies isn't feasible, we would have to think carefully if it's too risky to fix and then discard the original unfixed bag. (I think it might be acceptable if the procedure includes a thorough validation check script after fixing. The fix script attempts to change only specific entries in the bag metadata, and keep everything else including the raw message contents byte-identical with the original bag, which is a nice conservative approach.)

[bcoltin]

This is now included in the documentation. We will follow up with Jonathan to see about adding it to the default processing pipeline.

[bcoltin]

Superceded by more recent issue.