Be able to support compressed point cloud messages

[mlveggo]

Lidar point clouds continue to grow and visualizing lidar point in realtime can use a lot of data. Practical amount using for example 4 spinning 128 channel lidars on a vehicle will grow above 1Gb/s.

So I would propose to support some sort of compressed point cloud (gzip:ed, quantization) -> foxglove.CompressedPointCloud.

Discussion in Slack:
https://foxglove.slack.com/archives/C028UEY858S/p1650896371230199

[mlveggo]

Also added an issue to include this as a standard ROS message. ros2/common_interfaces#188

[Aposhian]

There are some messages and wrappers defined here: https://github.com/ros-perception/point_cloud_transport

[defunctzombie]

Another request for the same with some details on libraries to use: https://github.com/orgs/foxglove/discussions/907

[Aposhian]

I wrote a message converter extension from sensor_msgs/msg/PointCloud2 to point_cloud_transport/msg/CompressedPointCloud2. So far I have only implemented the zstd codec, but that works!

https://github.com/fireflyautomatix/foxglove-compressed-pointcloud

Ex:

ros2 bag play my_bag --topics /my_pointcloud_topic

ros2 run point_cloud_transport republish --in_transport raw --out_transport zstd --ros-args --remap in:=/my_pointcloud_topic

ros2 bag record -s mcap /out/zstd

And then loading that MCAP into Foxglove Studio, the /out/zstd topic is visualizable in the 3D panel.

[defunctzombie]

Awesome! What's the disk savings like for the message data compared to using compression on the mcap file itself? In your other discussion thread you noted that the messages can make for large mcap files - but mcap recording in ROS can do zstd compression on chunks of messages. Were you disabling this and ending up with large mcap files?

The savings on-the-wire for compressed point clouds would be valuable but I don't expect it to have much impact in a compressed mcap file but would like to see numbers on that.

[Aposhian]

That's a good point. I'll run a test.

[Aposhian]

Ok, I recorded some tests, with a source pointcloud at 10Hz, and recording bags for 10 seconds. Ex:

ros2 bag play my_pointcloud_bag --topics my_pointcloud_topic
ros2 run point_cloud_transport republish --in_transport raw --out_transport zstd --ros-args --remap in:=/my_pointcloud_topic

timeout 10s ros2 bag record ...

message type	bag compression mode	size
sensor_msgs/msg/PointCloud2	none	286MiB
point_cloud_interfaces/msg/CompressedPointCloud2	none	108MiB
sensor_msgs/msg/PointCloud2	file	110MiB
sensor_msgs/msg/PointCloud2	message	112MiB
point_cloud_interfaces/msg/CompressedPointCloud2	file	108MiB
point_cloud_interfaces/msg/CompressedPointCloud2	message	109MiB

So it looks like if you are using mcap compression, file or message, the type compression doesn't really help you, which makes sense. I think I didn't discover this before because I assumed mcap zstd compression was enabled by default when using ros2 bag record, but I guess it isn't! I was used to ros2 bag convert using zstd compression by default.

[Aposhian]

With the size of pointclouds in my application (single scans from a single sensor), zstd is actually better than the draco compression. But Draco says that their compression performs better with larger pointclouds, so in that case, maybe there is still a use-case for being able to display Draco CompressdPointCloud2 in Foxglove.

Specifically, one can expect much better compression rates for larger and denser point clouds.

https://github.com/google/draco/blob/master/README.md#encoding-point-clouds