Checklist for supporting document generation for ament_python packages

[Yadunund]

Automatic document generation for python (ament_python) packages in the ROS ecosystem has been a long pending ask and the purpose of this ticket is to serve as a checklist of PRs/patches to land to deliver this functionality.

PRs/Code to merge into this repo in specific order:

• Add support for python-only projects #28: Bulk of the required code is in this feature branch
• rkent@3fb7141: Port changes to sphinx_builder.py to append pkg source directory to system path when using autogenerated conf.py.
• Copy user files into docs_build so they can be referenced in conf.py #47
• TODO: Similar to Copy user files into docs_build so they can be referenced in conf.py #47, also copy package_src_directory made available from rkent@3fb7141 into docs_build such that the python interpreter can access pakg modules when running the user written conf.py file
• Yadunund@af64741: Correctly populate autodoc_mock_imports with python modules that are listed as system packages in package.xml (eg. python3-babeltrace).
• Yadunund@226f905: Include <doc_depend> dependencies in list of modules to include in autodoc_mock_imports
• Bonus: Update sphix_builder.py to not always overwrite any user specific autodoc_mock_imports with exec_depend modules.

I've incorporated all these changes into temporary branch for my testing and I'm pleased to share that document generation has gone well for most of the ament_python packages in the ros2 org. Along the way I've had to make some changes some packages to successfully generate documentation. Most of the changes are related to populating the package.xml with missing exec_depends. Without this, autodoc will try to import these dependency modules when importing the module to be documented. The downside of this is that cross-reference across pkgs is not yet supported but I'll discuss this and other todo's for the future later below. Other changes include fixes to docstrings to comply with sphinx/autodoc spec.

PRs to merge to generate documentation without any warnings

• Enable document generation using rosdoc2 for ament_python pkgs ros2/demos#606
• Enable document generation using rosdoc2 for ament_python pkgs ros2/examples#357
• Enable document generation using rosdoc2 for ament_python pkgs ros2/geometry2#587
• Enable document generation using rosdoc2 for ament_python pkgs ros2/launch#697
• Enable document generation using rosdoc2 for ament_python pkgs ros2/launch_ros#359
• Enable document generation using rosdoc2 for ament_python pkgs ros2/ros2cli#811
• Enable document generation using rosdoc2 for ament_python pkgs ros2/rosbag2#1260
• Enable document generation using rosdoc2 for ament_python pkgs ros2/ros2_tracing#50
• Enable document generation using rosdoc2 for ament_python pkgs ros2/rosidl#726
• Enable document generation using rosdoc2 for ament_python pkgs ros2/common_interfaces#217
• Enable document generation using rosdoc2 for ament_python pkgs ros2/sros2#284

Document generation results using combined changes

Pkg	Works?
action_tutorials_py	✅
demo_nodes_py	✅
domain_coordinator	✅
examples_rclpy_executors	✅
examples_rclpy_guard_conditions	✅
examples_rclpy_minimal_action_client	✅
examples_rclpy_minimal_action_server	✅
examples_rclpy_minimal_client	✅
examples_rclpy_minimal_publisher	✅
examples_rclpy_minimal_service	✅
examples_rclpy_minimal_subscriber	✅
examples_rclpy_pointcloud_publisher	✅
examples_tf2_py	✅
launch	✅
launch_pytest	✅
launch_ros	❌
launch_testing	✅
launch_testing_examples	✅
launch_testing_ros	✅
launch_xml	✅
launch_yaml	✅
lifecycle_py	✅
quality_of_service_demo_py	✅
rclpy	❌
ros2action	✅
ros2bag	✅
ros2cli	✅
ros2component	✅
ros2doctor	✅
ros2interface	✅
ros2launch	✅
ros2lifecycle	✅
ros2multicast	✅
ros2node	✅
ros2param	✅
ros2pkg	✅
ros2run	✅
ros2service	✅
ros2test	✅
ros2topic	✅
ros2trace	✅
rosidl_cli	✅
rosidl_pycommon	✅
rosidl_runtime_py	✅
rpyutils	❌
sensor_msgs_py	✅
sros2	✅
test_launch_ros	✅
test_tracetools_launch	✅
tf2_ros_py	✅
tf2_tools	✅
topic_monitor	✅
tracetools_launch	✅
tracetools_read	✅
tracetools_test	✅
tracetools_trace	✅

[ros-discourse]

This issue has been mentioned on ROS Discourse. There might be relevant details there:

https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2023-03-16/30432/1