Merge branch 'Farama-Foundation:main' into main

.pre-commit-config.yaml

@@ -39,7 +39,7 @@ repos:
         language: node
         pass_filenames: false
         types: [python]
-        additional_dependencies: ["pyright"]
+        additional_dependencies: ["pyright@1.1.347"]
         args:
           - --project=pyproject.toml
   - repo: https://github.com/pycqa/pydocstyle

README.md

@@ -32,6 +32,8 @@ The [D4RL](https://github.com/Farama-Foundation/D4RL) environments are now avail
 The different tasks involve hammering a nail, opening a door, twirling a pen, or picking up and moving a ball.
 * [Franka Kitchen](https://robotics.farama.org/envs/franka_kitchen/) - Multitask environment in which a 9-DoF Franka robot is placed in a kitchen containing several common household items. The goal of each task is to interact with the items in order to reach a desired goal configuration.
 
+* [MaMuJoCo](https://robotics.farama.org/envs/MaMuJoCo/) - A collection of multi agent factorizations of the [Gymnasium/MuJoCo](https://gymnasium.farama.org/environments/mujoco/) environments and a framework for factorizing robotic environments, uses the [pettingzoo.ParallelEnv](https://pettingzoo.farama.org/api/parallel/) API. 
+
 **WIP**: generate new `D4RL` environment datasets with [Minari](https://github.com/Farama-Foundation/Minari).
 
 ## Multi-goal API

docs/content/multi-goal_api.md

@@ -21,6 +21,9 @@ goal, e.g. state derived from the simulation.
 
 ```python
 import gymnasium as gym
+import gymnasium_robotics
+
+gym.register_envs(gymnasium_robotics)
 
 env = gym.make("FetchReach-v2")
 env.reset()

docs/envs/MaMuJoCo/index.md

@@ -19,20 +19,19 @@ Gymansium-Robotics/MaMuJoCo Represents the first, easy to use Framework for rese
 MaMuJoCo mainly uses the [PettingZoo.ParallelAPI](https://pettingzoo.farama.org/api/parallel/), but also supports a few extra functions:
 
 ```{eval-rst}
-.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_local_actions_to_global_action
+.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_local_actions_to_global_action
 ```
-
 ```{eval-rst}
-.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_global_action_to_local_actions
+.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_global_action_to_local_actions
 ```
 ```{eval-rst}
-.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_global_state_to_local_observations
+.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_global_state_to_local_observations
 ```
 ```{eval-rst}
-.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.map_local_observation_to_global_state
+.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.map_local_observations_to_global_state
 ```
 ```{eval-rst}
-.. autofunction:: gymnasium_robotics.mamujoco_v0.get_parts_and_edges
+.. autofunction:: gymnasium_robotics.mamujoco_v1.get_parts_and_edges
 ```
 
 MaMuJoCo also supports the [PettingZoo.AECAPI](https://pettingzoo.farama.org/api/aec/) but does not expose extra functions.
@@ -41,19 +40,19 @@ MaMuJoCo also supports the [PettingZoo.AECAPI](https://pettingzoo.farama.org/api
 
 ### Arguments
 ```{eval-rst}
-.. autofunction:: gymnasium_robotics.mamujoco_v0.parallel_env.__init__
+.. autofunction:: gymnasium_robotics.mamujoco_v1.parallel_env.__init__
 ```
 
 
 
 ## How to create new agent factorizations 
-### example 'Ant-v4', '8x1'
+### example 'Ant-v5', '8x1'
 In this example, we will create an agent factorization not present in Gymnasium-Robotics/MaMuJoCo the "Ant"/'8x1', where each agent controls a single joint/action (first implemented by [safe-MaMuJoCo](https://github.com/chauncygu/Safe-Multi-Agent-Mujoco)).
 
 first we will load the graph of MaMuJoCo:
 ```python
->>> from gymnasium_robotics.mamujoco_v0 import get_parts_and_edges
->>> unpartioned_nodes, edges, global_nodes = get_parts_and_edges('Ant-v4', None)
+>>> from gymnasium_robotics.mamujoco_v1 import get_parts_and_edges
+>>> unpartioned_nodes, edges, global_nodes = get_parts_and_edges('Ant-v5', None)
 ```
 The `unpartioned_nodes` contain the nodes of the MaMuJoCo graph.
 The `edges` well, contain the edges of the graph.
@@ -63,18 +62,23 @@ To create our '8x1' partition we will need to partition the `unpartioned_nodes`:
 ```python
 >>> unpartioned_nodes
 [(hip1, ankle1, hip2, ankle2, hip3, ankle3, hip4, ankle4)]
->>> partioned_nodes = [(unpartioned_nodes[0][0],), (unpartioned_nodes[0][1],), (unpartioned_nodes[0][2],), (unpartioned_nodes[0][3],), (unpartioned_nodes[0][4],), (unpartioned_nodes[0][5],), (unpartioned_nodes[0][6],), (unpartioned_nodes[0][7],)]>>> partioned_nodes
+>>> partioned_nodes = [(unpartioned_nodes[0][0],), (unpartioned_nodes[0][1],), (unpartioned_nodes[0][2],), (unpartioned_nodes[0][3],), (unpartioned_nodes[0][4],), (unpartioned_nodes[0][5],), (unpartioned_nodes[0][6],), (unpartioned_nodes[0][7],)]
 >>> partioned_nodes
 [(hip1,), (ankle1,), (hip2,), (ankle2,), (hip3,), (ankle3,), (hip4,), (ankle4,)]
 ```
 Finally package the partitions and create our environment:
 ```python
 >>> my_agent_factorization = {"partition": partioned_nodes, "edges": edges, "globals": global_nodes}
->>> gym_env = mamujoco_v0('Ant', '8x1', agent_factorization=my_agent_factorization)
+>>> gym_env = mamujoco_v1('Ant', '8x1', agent_factorization=my_agent_factorization)
 ```
 
 ## Version History
-v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of [the original multiagent_mujuco](https://github.com/schroederdewitt/multiagent_mujoco)
+* v1:
+	- Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572).
+	- When `factorizatoion=None`, the `env.gent_action_partitions.dummy_node` now contains `action_id` (it used to be `None`).
+	- Added `map_local_observations_to_global_state` & optimized runtime performance of `map_global_state_to_local_observations`.
+	- Added `gym_env` argument which can be used to load third-party `Gymansium.MujocoEnv` environments.
+* v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of [the original multiagent_mujuco](https://github.com/schroederdewitt/multiagent_mujoco)
 
 ```{toctree}
 :hidden:
@@ -84,6 +88,7 @@ ma_half_cheetah.md
 ma_hopper.md
 ma_humanoid_standup.md
 ma_humanoid.md
+ma_multiagentswimmer.md
 ma_reacher.md
 ma_swimmer.md
 ma_pusher.md

docs/envs/MaMuJoCo/ma_ant.md

@@ -15,22 +15,22 @@ This Environment is part of [MaMuJoCo](https://robotics.farama.org/envs/MaMuJoCo
 The task is [Gymansium's MuJoCo/Ant](https://gymnasium.farama.org/environments/mujoco/ant/).
 
 
-| Defaults		| `env = mamujoco_v0.parallel_env("Ant", None)`		|
+| Defaults		| `env = mamujoco_v1.parallel_env("Ant", None)`		|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0']`					|
 | Number of Agents	| 1							|
 | Action Spaces		| `{'agent_0' : Box(-1, 1, (8,), float32)}`			|
 | Part partition	| `[(hip4, ankle4, hip1, ankle1, hip2, ankle2, hip3, ankle3)]`	|
 
 ## Action Space
-The action spaces is depended on the partitioning
+The shape of the action space depends on the partitioning. The partitioning has the following form:
 
 ### if partitioning is None:
 ```{figure} figures/ant.png
     :name: ant
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Ant", None)`		|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Ant", None)`		|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0']`					|
 | Number of Agents	| 1							|
@@ -56,7 +56,7 @@ If partitioning, is None then the environment contains a single agent with the s
     :name: ant_2x4
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Ant", "2x4")`		|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Ant", "2x4")`		|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1']`					|
 | Number of Agents	| 2							|
@@ -86,7 +86,7 @@ The environment is partitioned in 2 parts, the front part (containing the front
     :name: ant_2x4d
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Ant", "2x4d")`		|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Ant", "2x4d")`		|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1']`					|
 | Number of Agents	| 2							|
@@ -114,7 +114,7 @@ The environment is partitioned in 2 parts, split diagonally.
     :name: ant_4x2
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Ant", "4x2")`		|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Ant", "4x2")`		|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1', 'agent_2', 'agent_3']`			|
 | Number of Agents	| 4							|
@@ -168,7 +168,7 @@ All agents receive the same [Gymnasium's Ant](https://gymnasium.farama.org/envir
 
 
 ## Starting state
-The starting state of the environment is the as [Gymnasium's Ant](https://gymnasium.farama.org/environments/mujoco/ant/#starting-state).
+The starting state of the environment is the same as [Gymnasium's Ant](https://gymnasium.farama.org/environments/mujoco/ant/#starting-state).
 
 
 
@@ -178,7 +178,11 @@ All agent terminate and truncate at the same time given the same conditions as [
 
 
 ## Version History
-- v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco).
+* v1:
+	- Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572).
+	- Now observes `local_categories` of `cfrc_ext` by default (same as `Gymnasium/MuJoCo-v5/Ant`).
+	- Renamed global node `torso` → `root`.
+* v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco).
 Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)):
 	- Fixed diagonal factorization ("2x4d") not being diagonal.
 	- Fixed Global observations (The Ant's Torso: `rootx`, `rooty`, `rootz`) not being observed.

docs/envs/MaMuJoCo/ma_coupled_half_cheetah.md

@@ -11,14 +11,14 @@ The environment consists of 2 half cheetahs coupled by an elastic tendon.
 
 
 ## Action Space
-The action spaces is depended on the partitioning.
+The shape of the action space depends on the partitioning. The partitioning has the following form:
 
 ### if partitioning is None:
 ```{figure} figures/coupled_half_cheetah.png
     :name: coupled_half_cheetah
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("CoupledHalfCheetah", None)`	|
+| Instantiate		| `env = mamujoco_v1.parallel_env("CoupledHalfCheetah", None)`	|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0']`					|
 | Number of Agents	| 1							|
@@ -49,7 +49,7 @@ If partitioning, is `None`, then the environment contains a single agent with th
     :name: coupled_half_cheetah_1p1
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("CoupledHalfCheetah", "1p1")`|
+| Instantiate		| `env = mamujoco_v1.parallel_env("CoupledHalfCheetah", "1p1")`|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1']`			|
 | Number of Agents	| 2							|
@@ -98,7 +98,7 @@ All agents receive the same average reward of each cheetah.
 
 
 ## Starting state
-The starting state of the environment is the as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state) (but with 2 cheetahs).
+The starting state of the environment is the same as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state) (but with 2 cheetahs).
 
 
 
@@ -108,6 +108,8 @@ All agent terminate and truncate at the same time, given the same conditions as
 
 
 ## Version History
+* v1:
+	- Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572).
 - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco).
 Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)):
 	- Fixed action mapping of the second cheetah (It would previously not work)

docs/envs/MaMuJoCo/ma_half_cheetah.md

@@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Half Cheetah](https://gymnasium.farama.org/envir
 
 
 ## Action Space
-The action spaces is depended on the partitioning
+The shape of the action space depends on the partitioning. The partitioning has the following form:
 
 ### if partitioning is None:
 ```{figure} figures/half_cheetah.png
     :name: half_cheetah
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("HalfCheetah", None)`	|
+| Instantiate		| `env = mamujoco_v1.parallel_env("HalfCheetah", None)`	|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0']`					|
 | Number of Agents	| 1							|
@@ -47,7 +47,7 @@ If partitioning, is `None`, then the environment contains a single agent with th
     :name: half_cheetah_2x3
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("HalfCheetah", "2x3")`|
+| Instantiate		| `env = mamujoco_v1.parallel_env("HalfCheetah", "2x3")`|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1']`					|
 | Number of Agents	| 2							|
@@ -75,7 +75,7 @@ The environment is partitioned in 2 parts, the front part (containing the front
     :name: half_cheetah_6x1
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("HalfCheetah", "6x1")`|
+| Instantiate		| `env = mamujoco_v1.parallel_env("HalfCheetah", "6x1")`|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1', 'agent_2', 'agent_3', 'agent_4', 'agent_5']`			|
 | Number of Agents	| 6							|
@@ -129,7 +129,7 @@ All agents receive the same [Gymnasium's Half Cheetah](https://gymnasium.farama.
 
 
 ## Starting state
-The starting state of the environment is the as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state).
+The starting state of the environment is the same as [Gymnasium's Half Cheetah](https://gymnasium.farama.org/environments/mujoco/half_cheetah/#starting-state).
 
 
 
@@ -138,6 +138,8 @@ All agent terminate and truncate at the same time, given the same conditions as
 
 
 ## Version History
+* v1:
+	- Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572).
 - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco).
 Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)):
 	- Added/Fixed Global observations (The Cheetah's front tip: `rootx`, `rooty`, `rootz`) not being observed.

docs/envs/MaMuJoCo/ma_hopper.md

@@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Hopper](https://gymnasium.farama.org/environment
 
 
 ## Action Space
-The action spaces is depended on the partitioning
+The shape of the action space depends on the partitioning. The partitioning has the following form:
 
 ### if partitioning is None:
 ```{figure} figures/hopper.png
     :name: hopper
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Hopper", None)`	|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Hopper", None)`	|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0']`					|
 | Number of Agents	| 1							|
@@ -47,7 +47,7 @@ If partitioning, is `None`, then the environment contains a single agent with th
     :name: hopper_3x1
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Hopper", "3x1")`|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Hopper", "3x1")`|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1', 'agent_2']`			|
 | Number of Agents	| 3							|
@@ -89,7 +89,7 @@ All agents receive the same [Gymnasium's Hopper](https://gymnasium.farama.org/en
 
 
 ## Starting state
-The starting state of the environment is the as [Gymnasium's Hopper](https://gymnasium.farama.org/environments/mujoco/hopper/#starting-state).
+The starting state of the environment is the same as [Gymnasium's Hopper](https://gymnasium.farama.org/environments/mujoco/hopper/#starting-state).
 
 
 
@@ -98,6 +98,8 @@ All agent terminate and truncate at same time given the same conditions as [Gymn
 
 
 ## Version History
+* v1:
+	- Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572).
 - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco).
 Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)):
 	- Fixed Global observations (The Hopper's top: `rootx`, `rooty`, `rootz`) not being observed.

docs/envs/MaMuJoCo/ma_humanoid.md

@@ -17,14 +17,14 @@ The task is [Gymansium's MuJoCo/Humanoid](https://gymnasium.farama.org/environme
 
 
 ## Action Space
-The action spaces is depended on the partitioning
+The shape of the action space depends on the partitioning. The partitioning has the following form:
 
 ### if partitioning is None:
 ```{figure} figures/humanoid.png
     :name: humanoid
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Humanoid", None)`	|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Humanoid", None)`	|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0']`					|
 | Number of Agents	| 1							|
@@ -60,7 +60,7 @@ If partitioning, is `None` then the environment contains a single agent with the
     :name: humanoid_9|8
 ```
 
-| Instantiate		| `env = mamujoco_v0.parallel_env("Humanoid", "3x1")`|
+| Instantiate		| `env = mamujoco_v1.parallel_env("Humanoid", "3x1")`|
 |-----------------------|------------------------------------------------------|
 | Agents		| `agents= ['agent_0', 'agent_1']`			|
 | Number of Agents	| 2							|
@@ -114,7 +114,7 @@ All agents receive the same [Gymnasium's Humanoid](https://gymnasium.farama.org/
 
 
 ## Starting state
-The starting state of the environment is the as [Gymnasium's Humanoid](https://gymnasium.farama.org/environments/mujoco/humanoid/#starting-state).
+The starting state of the environment is the same as [Gymnasium's Humanoid](https://gymnasium.farama.org/environments/mujoco/humanoid/#starting-state).
 
 
 
@@ -123,6 +123,9 @@ All agent terminate and truncate at the same time, given the same conditions as
 
 
 ## Version History
+* v1:
+	- Now based on `Gymnasium/MuJoCo-v5` instead of `Gymnasium/MuJoCo-v4` (https://github.com/Farama-Foundation/Gymnasium/pull/572).
+	- No longer observes `qfrc_actuator` of `root` & `cinert`, `cvel`, `qfrc_actuator`, `cfrc_ext` of `worldbody` (same as `Gymnasium/MuJoCo-v5/Humanoid`).
 - v0: Initial version release, uses [Gymnasium.MuJoCo-v4](https://gymnasium.farama.org/environments/mujoco/), and is a fork of the original MaMuJoCo [schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco).
 Changes from the original `MaMuJoCo` ([schroederdewitt/multiagent_mujoco](https://github.com/schroederdewitt/multiagent_mujoco)):
 	- Added/Fixed Global observations (The Humanoids's torso: `rootx`, `rooty`, `rootz`) not being observed.