Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multirobot SLAM ros2 #592

Open
wants to merge 17 commits into
base: ros2
Choose a base branch
from

Conversation

acachathuranga
Copy link


Basic Info

An extension of slam_toolbox for ROS2 multi-robot distributed SLAM. This work was done as part of a research at Singapore University of Technology and Design.

Info Please fill out this column
Ticket(s) this addresses (76)
Primary OS tested on (Ubuntu 22.04)
Tested on gazebo and indoor space at Singapore University of Technology and Design Gazebo and Custom made Physical Robot (Monsterborg platform with RPLidar A3)

Description of contribution in a few bullet points

  • Extends slam_toolbox to perform multi-robot mapping and localization in ROS2 (Can also be used for multi-laser scenarios).
  • multirobot_slam_toolbox_node implements the new functionality separately (Behaviors and functionalities of all other slam_toolbox nodes remain unchanged).
  • Reduces network bandwidth usage by only sharing pose-graph scan data between each robot (Uses new message type LocalizedLaserScan). No sharing of any other topics or data between the robots is required.
  • Constructs a merged pose-graph and performs loop closure distributively on each robot.
  • Not the original use-case, but this can also be used in multi-laser robots, by running a slam_toolbox instance for each laser.

multi-robot_slam

Description of documentation updates required from your changes

  • Slam_toolbox instances needs to be run on each robot under unique namespaces.
  • New topic '/localized_scan' to share pose-graph data between robots. This topic must not be in a namespace.
  • Each slam_toolbox instance publishes transform of host robot and peers in the network. Peer transform frame names are prefixed with the namespace of the peer.
  • Robots must start close to each other, in the same orientation. Scan matching is used to get the relative pose between the two pose graphs.
  • Each laser is named by the host namespace.

multi-robot_architecture


Future work that may be required in bullet points

  • Take the initial start pose of robots through input arguments / parameters (without requiring robots to start closer to each other)
  • Experiment on lossy WiFi networks. How would the nodes handle scan message losses between each others.

@acachathuranga acachathuranga changed the title Multirobot ros2 Multirobot SLAM ros2 Mar 30, 2023
@Daniel-Meza
Copy link

@acachathuranga this looks great.

Could you go into more detail about how you set up the two robot scenario and run the multirobot_slam_toolbox_node?
Looking at your forked multirobot_ros2 repo I don't see launch files to use as reference. If you have the ones you used for the demo above that would also be very helpful!

FYI, I am trying to replicate your demo using two Turtlebot3 robot platforms in Gazebo. I have set up the namespaces according to your diagram but when launching everything I can't seem to get the correct /robot1/map and /robot2/map topics to publish and the /localized_scan topic shows no published messages.

Appreciate any pointers.

@acachathuranga
Copy link
Author

@Daniel-Meza I have added the launch and parameter files for multi-robot mapping. The parameters file is identical to mapper_params_online_async.yaml.

I have also updated the ReadMe on how to run multi-robot slam with turtlebot3 multirobot simulation (provided with navigation2 nav2_bringup package).

In short, you can follow the installation instructions in navigation2 Getting started tutorial. Once you've installed navigation2, nav2_bringup and turtlebot3 packages, source the environment variables as described and run the following commands in separate terminals.

ros2 launch nav2_bringup multi_tb3_simulation_launch.py headless:=False
ros2 launch slam_toolbox online_async_multirobot_launch.py namespace:=robot1
ros2 launch slam_toolbox online_async_multirobot_launch.py namespace:=robot2

Then you can use teleop_twist_keyboard to drive the robots around using following commands. When driving the robots around, the merged map will be displayed on both rviz windows.

ros2 run teleop_twist_keyboard teleop_twist_keyboard __ns:=/robot1
ros2 run teleop_twist_keyboard teleop_twist_keyboard __ns:=/robot2

@Daniel-Meza
Copy link

@acachathuranga Thank you, I was able to replicate the demo and also get it working with a 3rd robot.

Going a bit further, do you have some insights on the following:

Maximum number of robots. Since every robot/sensor runs its own slam_toolbox node, do you foresee any limitation on the number of robots that can be used simultaneously? Assuming of course there is enough computing power and bandwidth to handle all of the data.

Running on real robots. In Gazebo simulation, I believe the "map" and "odom" frame is always placed at a global (0, 0, 0) regardless of where the robots are spawned. This means that every slam_toolbox node has their "map" located based on the same initial point. This is not the case when booting up real robots since the map frame is placed on the current physical location of the robot itself. So, would it be necessary to provide an additional transform to ensure that the "map" frame of real robots align on top of each other? Or is this transformation handled automatically by the multirobot nodes?

@acachathuranga
Copy link
Author

acachathuranga commented Apr 11, 2023

@Daniel-Meza nice to hear you got it working!

Maximum number of robots: Given there is enough computing power and bandwidth, I do not see any limitation on the number of robots (Unless there are underlying SLAM limitations within slam_toolbox on number of graph nodes etc). But nothing specific to 'multi-robot slam'. However, when the robot count increases in real world scenarios (at least during our testing), you're likely to run into bandwidth problems before anything else :)

Running on real robots: This is explained in the PR notes as well as the updated ReadMe. To summarize; gazebo simulation odometry is indeed in a global frame. So everything will automatically work. When you run on actual robots, you need to start the robots very close to each other, and also in the same orientation. Then the scan matcher will handle the initial offsets. In our testing, what we did was, 1) turn on one robot at a marked location, 2) move the robot away from the location 3) place another robot at the marked location and repeat. Taking initial poses from input args /parameters will be added in near future.

@i1Cps
Copy link

i1Cps commented Apr 27, 2024

Good Morning, Are you able to incoperate the recently added /reset feature with the multi-robot solution?

@acachathuranga
Copy link
Author

@SteveMacenski any thoughts or comments on this? I noticed many people find this multi-robot implementation useful and there have also been some recent duplicate efforts (PR #727) on this multi-robot slam topic.

This work has been tested extensively with real robot fleets (2 robot, 4 robot, 8 robot configurations) and has provided very good results. The use of '/localized_scan' topic instead of sharing all laser scans among all robots provides real-life benefits in terms of network bandwidth. We also did extensive tests with different types of network (Infrastructure based, mesh .etc) and found out sharing all laser messages is indeed not practical for real situations. We already have done further extensions on this work, in terms of bandwidth optimizations for large robot fleets. I will push these updates once our work is published.

@MikeDegany
Copy link

Is it possible to share robots initial poses (relative poses) before multi-robot mapping starts, so that the robots do not need to start from the same location?

@SteveMacenski
Copy link
Owner

SteveMacenski commented Nov 6, 2024

Sorry for the delays here - this is something I'm going to prioritize responses and a resolution to now. Expect more responses from me now :-)

There's another PR open by @Daniel-Meza which adds this into humble in a different way (ie the nodes themselves have multirobot support added instead of adding in a separate node) in https://github.com/UTNuclearRoboticsPublic/slam_toolbox/tree/humble_multirobot_v2.

I would like to setup a call with Daniel, you @acachathuranga and myself to chat about steps forward here so we can get some solution into ros2 and possibly backported into Humble / Jazzy. We have three suggested ways (this #727 and the branch I liked above) and I'd like for us to pick the way we want to go; each test it for our needs; review it so that we're all happy and merge for a release. I think its easier to do that with a synchronous meeting so there's no delay.

I possibly see a solution being 2 fold: your new node here and Daniel's work to embed the optionality into the single node for less networking complex situations / simulation. Or if we all agree one way in particular is the right way to go for all cases, then we can go that route and work together to make that happen.

@Daniel-Meza
Copy link

Sounds good, @SteveMacenski.
I can usually find time mid-day during the week. If you could suggest a few meeting times in the coming days, we can get something scheduled!

@SteveMacenski
Copy link
Owner

Mid day in which time zone 😉

@acachathuranga what about you?

@Daniel-Meza
Copy link

Good point 😅
Between 11am - 4pm CDT

@acachathuranga
Copy link
Author

Sorry for the late reply! Thanks for your time on this @SteveMacenski. I'm 12hrs ahead of your timezone haha, but actually I'm okay with any time this month. Let me know a convenient time for you, thanks!

@SteveMacenski
Copy link
Owner

@Daniel-Meza could you do 10am CDT? That would be 8am my time, 10am your time, and 8pm @acachathuranga time -- which seems like the most reasonable cross section for everyone involved if you could do a little earlier. I'd be open Nov 20, 22, 26 (to name a few)

@Daniel-Meza
Copy link

Yes, I can make 10am CDT on Nov 20 or 26.

@SteveMacenski
Copy link
Owner

@Daniel-Meza you can find my email on my github page. Please email me so I have your email to set up the meeting. I see @acachathuranga has an email on his account that I can use

@SteveMacenski
Copy link
Owner

Invites were sent out!

@SteveMacenski
Copy link
Owner

Notes from our conversation:

  • Review this PR, make adjustments as needed
  • @acachathuranga add documentation on the initialization methods in detail
  • @Daniel-Meza review it and test it
  • Steve: test it
  • Obtain gif of robots / slam
  • Merge baby merge 🥳

Copy link
Owner

@SteveMacenski SteveMacenski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like this PR is in conflict with ros2 now, that also needs to be resolved before I/Daniel can test. I also see a few linting issues, running ament_cpplint and ament_uncrustify will reveal them!

@@ -64,6 +64,7 @@ set(libraries
toolbox_common
SlamToolboxPlugin
ceres_solver_plugin
multirobot_slam_toolbox
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this + the Class + file names: I think we need to make this more specific since we'll have both nodes in the project.

Suggestion: Decentralized Multi-robot?


*multirobot_slam_toolbox_node* extends slam_toolbox for multi-robot mapping and localization. Slam_toolbox instances are run on each robot under unique namespaces and pose graph data are shared between the nodes through */localized_scan* topic. Each slam_toolbox instance publishes transform of host robot and peers in the network. Peer transform frame names are prefixed with the namespace of the peer.

Initially the robots should start close to each other, in the same orientation, as scan matching is used to find the connection between different pose graphs. This however is not required in simulation environments such as gazebo, as all robots' odometry is referenced to a gazebo's global frame.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed: expand on this. If you can show an example project setting up the shared frame, that would be immensely helpful for those that want to use your work

# Multi-Robot SLAM
![multirobot_slam](/images/multi-robot_mapping.gif?raw=true "Multi-Robot SLAM")

*multirobot_slam_toolbox_node* extends slam_toolbox for multi-robot mapping and localization. Slam_toolbox instances are run on each robot under unique namespaces and pose graph data are shared between the nodes through */localized_scan* topic. Each slam_toolbox instance publishes transform of host robot and peers in the network. Peer transform frame names are prefixed with the namespace of the peer.
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd expand on this description about how the data is shared (localized scans published once processed by a robot's host node to sync up with others), why (network limitations). Enough so that when someone reads the software, they know what theyre looking at

@@ -80,10 +80,12 @@ class LaserAssistant
const std::string & base_frame);
~LaserAssistant();
LaserMetadata toLaserMetadata(sensor_msgs::msg::LaserScan scan);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the single version be removed and uses switched to the new one?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the single version just wraps the multi-version now

@@ -144,34 +153,39 @@ karto::LaserRangeFinder * LaserAssistant::makeLaser(const double & mountingYaw)

bool LaserAssistant::isInverted(double & mountingYaw)
{
geometry_msgs::msg::TransformStamped laser_ident;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why was this method updated?

: SlamToolbox(options), localized_scan_topic_("/localized_scan")
/*****************************************************************************/
{
current_ns_ = this->get_namespace() + 1;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get_namespace returns a string, I'm not sure what this intends to do

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After reviewing the file, the use of current_ns_ and its modification to the published localized range scan should be documented / explained further


/*****************************************************************************/
MultiRobotSlamToolbox::MultiRobotSlamToolbox(rclcpp::NodeOptions options)
: SlamToolbox(options), localized_scan_topic_("/localized_scan")
Copy link
Owner

@SteveMacenski SteveMacenski Nov 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a parameter (the topic to use)


sensor_msgs::msg::LaserScan::ConstSharedPtr scan =
std::make_shared<sensor_msgs::msg::LaserScan>(localized_scan->scan);
Pose2 pose;
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a x,y,heading constructor for Pose2 that would make this more concise https://github.com/SteveMacenski/slam_toolbox/blob/ros2/lib/karto_sdk/include/karto_sdk/Karto.h#L2063

}
LocalizedRangeScan * range_scan = addExternalScan(laser, scan, pose);

if (range_scan != nullptr)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also publishing - aren't we having some confliction if this is also publishing?

https://github.com/SteveMacenski/slam_toolbox/blob/ros2/src/slam_toolbox_common.cpp#L472-L498

@acachathuranga
Copy link
Author

acachathuranga commented Nov 29, 2024

Sorry, it had been a very busy couple weeks trying to wrap up work things before holidays, so couldn't finish all the changes. But the linting issues and the conflicts with ros2 branch are fixed now.
Yet to be done,

  • Update documentation for new multi-robot launch files in nav2-bringup
    The new humble binaries for multi-robot launch in nav2-bringup seems to have some issue with parameters(Iron/Jazzy parameters back-ported humble I guess?). So, need to sort out how to guide others to easily run and play around with a multi-robot sim + slam.
  • Complete review changes

Sorry again for the delays as I'll be traveling starting tomorrow, so will take a while for me to finish the items. But will try my best to finish all before new year!

@SteveMacenski
Copy link
Owner

No worries at all, its a busy time of year :-) Let us know when you're ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants