Scripting: plugins that aren't written in C++

[chapulina]

Desired behavior

In the interest of making Gazebo more approachable and quicker to use, we could offer a scripting interface that lets users write plugins in Python (and potentially other languages too, like Ruby and Javascript).

The user could load their Python script through the SDF's <plugin> tag, just like they do with C++ libraries right now.

Alternatives considered

Users could use ROS to interact with Gazebo using various scripting languages. We could also just offer a scripting API to Ignition Transport, so users can interact with simulation asynchronously.

These 2 approaches limit what the user can do though, because they're constrained to the transport API and must run async.

Implementation suggestion

First we'll need to expose C++ APIs that are commonly used by plugins to scripts, such as:

• The EntityComponentManager's API so users can access and modify entities and components
• Component data types so that they can be read and manipulated. Components commonly include Math, SDFormat and Msgs types.
• All the System callbacks need to be inherited by the Python plugin so that they're called at the correct time during simulation.
  • Note that in C++, systems receive a mutable reference to the ECM through these callbacks, and that's how they modify components and entities. This may be complicated to do when interfacing with Python.

We can either add C APIs to all of these and then call those C functions from scripts, or use SWIG to generate APIs in other languages from C++.

Finally, we'll need to support loading those scripts from the <plugin> tag.

Additional context

Related to #789

[chapulina]

The EntityComponentManager's API so users can access and modify entities and components

An alternative idea is to only provide python bindings for the simpler wrapper classes, like World, Model and Link, see #325. The rationale is:

• The scripting interface is geared towards beginners, so we should expose our simpler APIs. Advanced users who'd directly use the ECM should be able to, and most likely will choose to, use C++.
• I think it may be simpler to expose those classes and their simpler setters and getters, than to expose the ECM and all the complexity that comes with templates, component creation, etc.

[diegoferigo]

A possible implementation of plugins that can be developed in Python is exploiting the functional features of pybind11. I've never tried to do it myself, but it is maybe possible to develop toolings comparable to #926 to define a system with custom-defined callbacks implemented in Python

Example

class MyPythonSystem:
    def __init__(server: gazebo.Server):

        # Created from the ECM in the configuring phase
        self.model: Optional[gazebo.Model] = None
        
        # Custom system that allows registering callbacks
        helper_system = gazebo.HelperSystem()
        helper_system.configure_callback = functools.partial(MyPythonSystem.on_configure, self=self)
        helper_system.pre_update_callback = functools.partial(MyPythonSystem.on_pre_update, self=self)

        # Insert the system programmatically
        if not server.add_system(system=helper_system, world_idx=0):
            raise RuntimeError("Failed to insert helper system")

    def on_configure(
        self,
        entity: gazebo.Entity,
        sdf_element: sdf.Element,
        ecm: gazebo.EntityComponentManager,
        event_manager: gazebo.EventManager) -> None:

        # Create and store the model
        self.model = gazebo.Model(entity)
        assert self.model.valid()

    def on_pre_update(
        self,
        update_info: gazebo.UpdateInfo, 
        ecm: gazebo.EntityComponentManager) -> None:

        print(self.model.name())
        # TODO other logic calling Model methods


server_config = ServerConfig()
server_config.set_sdf_file(sdf_file='rolling_shapes.sdf')
server = Server(server_config=server_config)

my_system = MyPythonSystem(server=server)
assert server.run(blocking=True, iterations=1, paused=True)

# Now running the server should also run MyPythonSystem.on_pre_update
assert server.run(blocking=True, iterations=1, paused=False)

# Calling methods from here should also be possible, exposing post-update ECM data
print(my_system.model.name())

[osrf-triage]

This issue has been mentioned on Gazebo Community. There might be relevant details there:

https://community.gazebosim.org/t/ignition-python-wrappers-released-in-fortress/1332/3

[Kakcalu13]

This is so perfect!

I'm so thrilled to see this! I have a quick question, would this be on Fortress only or would it include Citadel too? 👼

[chapulina]

Hi @Kakcalu13 , we currently have no plans to work on this in the coming months. This ticket is here to discuss possible approaches and gather feedback from the community. Glad to know there's interest! We also accept contributions 😄