Add native MarlGrid environments with prestige coloring

.github/PULL_REQUEST_TEMPLATE.md

@@ -39,3 +39,15 @@ If you are unsure about any of these, don't hesitate to ask. We are here to help
 - [ ] I have reformatted the code using `make format` (**required**)
 - [ ] I have checked the code using `make lint` (**required**)
 - [ ] I have ensured `make bazel-test` pass. (**required**)
+
+## New Environment Checklist
+
+For PRs that add a new environment family or new upstream task family:
+
+- [ ] Runtime logic is native C++ and does not bridge to the official Python environment.
+- [ ] All intended upstream task IDs/scenarios are registered, documented, and covered by tests.
+- [ ] The upstream oracle/version is pinned, and tests check EnvPool registration/configs against it.
+- [ ] Determinism tests cover reset plus multi-step rollouts for every registered ID, including render frames when rendering is supported.
+- [ ] Oracle alignment tests compare step-level observations, rewards, done/truncation, info, and renders after at most one reset-time state sync.
+- [ ] Render tests cover reset, multi-step, batched render/env-id selection, and docs include EnvPool-vs-official images when an official renderer exists.
+- [ ] `envpool/make_test.py`, release packaging, docs, and README support lists are updated.

README.md

@@ -18,6 +18,7 @@
 - [x] [Google Research Football](https://envpool.readthedocs.io/en/latest/env/gfootball.html)
 - [x] [Procgen](https://envpool.readthedocs.io/en/latest/env/procgen.html)
 - [x] [Minigrid](https://envpool.readthedocs.io/en/latest/env/minigrid.html)
+- [x] [MarlGrid](https://envpool.readthedocs.io/en/latest/env/marlgrid.html)
 - [x] [Highway](https://envpool.readthedocs.io/en/latest/env/highway.html)
 - [x] [MetaWorld](https://envpool.readthedocs.io/en/latest/env/metaworld.html)
 - [x] [MyoSuite](https://envpool.readthedocs.io/en/latest/env/myosuite.html)

docs/content/new_env.rst

@@ -839,3 +839,35 @@ Make Tests
 
 You can add a test in ``envpool/make_test.py`` to see if the environment can be
 successfully created.
+
+
+New Environment Review Checklist
+--------------------------------
+
+Before opening a PR for a new environment family, make sure the implementation
+is complete end-to-end:
+
+- The runtime implementation is native C++; do not call or embed the official
+  Python environment from C++ runtime code.
+- Pin the exact upstream oracle version when one exists, and keep all tests
+  anchored to that version.
+- Register every intended upstream task ID or scenario. Do not collapse multiple
+  upstream IDs into one generic EnvPool task.
+- Add registry coverage that checks EnvPool task IDs and task configuration
+  values against the pinned upstream source when practical.
+- Add deterministic tests that replay the same external action sequence across
+  reset plus nontrivial multi-step rollouts for every registered ID. If render
+  is supported, include rendered frames in the determinism check.
+- Add step-level oracle alignment tests when an official implementation exists.
+  A reset-time state sync is acceptable when needed, but do not sync state
+  again during the rollout. Compare observations, rewards, terminated/truncated
+  semantics, exposed info, and renders when render output is expected to match.
+- Add render tests for reset frames, multi-step frames, batched rendering, and
+  env-id selection when rendering is supported.
+- Add the new family to ``envpool/make_test.py`` so source builds and installed
+  release wheels exercise ``envpool.make_*`` for the registered import path.
+- Update the environment docs, docs index, README support list, and release
+  packaging. If an official renderer exists, add an EnvPool-vs-official render
+  comparison image to the environment doc page.
+- Keep tolerances narrow, platform-scoped, and documented. Do not skip new
+  environment tests on a supported platform just to make CI pass.

docs/env/marlgrid.rst

@@ -0,0 +1,99 @@
+MarlGrid
+========
+
+We use ``kandouss/marlgrid`` commit
+``e88c40bad07653575ac11fe2f3a115e4de3d13e9`` as the reference implementation.
+MarlGrid is a multi-agent gridworld based on the original MiniGrid codebase.
+Its GoalCycle environments were used in the 2021 paper by Ndousse and
+coauthors, `Emergent Social Learning via Multi-agent Reinforcement Learning
+<https://proceedings.mlr.press/v139/ndousse21a.html>`_.
+
+.. image:: ../_static/render_samples/marlgrid_official_compare.png
+   :align: center
+
+
+Options
+-------
+
+* ``task_id (str)``: see the available tasks below;
+* ``num_envs (int)``: how many environments you would like to create;
+* ``batch_size (int)``: the expected batch size for returned environments,
+  default to ``num_envs``;
+* ``num_threads (int)``: the maximum thread number for executing the actual
+  ``env.step``, default to ``batch_size``;
+* ``seed (int | Sequence[int])``: the environment seed. When a sequence is
+  provided, it must contain exactly one seed per environment. Default to
+  ``42``;
+* ``max_num_players (int)``: maximum number of players in one environment.
+  Each registered task defaults this to its number of agents;
+* ``prestige_coloring (bool)``: use the ``kandouss/marlgrid`` prestige cue for
+  agent rendering. This option defaults to ``False`` to preserve the fixed
+  agent colors used by existing tasks. When enabled, positive rewards move an
+  agent color from red toward blue, negative rewards reset it to red, and
+  prestige decays after each active agent step;
+* ``prestige_beta (float)``: per-step prestige decay factor, default to
+  ``0.95``;
+* ``prestige_scale (float)``: reward-history scale used when mapping prestige
+  to color, default to ``2.0``.
+
+
+Observation Space
+-----------------
+
+MarlGrid returns one RGB partial-view image per player. The default registered
+tasks use ``view_tile_size=8`` and expose ``obs`` as a uint8 tensor with shape
+``(view_tile_size * view_size, view_tile_size * view_size, 3)`` per player.
+
+Player metadata is returned under ``info["players"]``:
+
+* ``id``: player index inside the environment;
+* ``done``: per-player completion flag;
+* ``active``: whether the player currently renders and acts;
+* ``pos``: player position in the full grid;
+* ``dir``: player direction in ``[0, 3]``.
+
+
+Action Space
+------------
+
+Actions are per-player discrete values in ``[0, 6]``:
+
+* ``0``: turn left;
+* ``1``: turn right;
+* ``2``: move forward;
+* ``3``: pick up;
+* ``4``: drop;
+* ``5``: toggle / interact;
+* ``6``: done.
+
+Multi-agent tasks accept EnvPool's player-shaped action format, for example:
+
+.. code-block:: python
+
+   import envpool
+   import numpy as np
+
+   env = envpool.make_gymnasium("MarlGrid-3AgentCluttered11x11-v0", num_envs=2)
+   obs, info = env.reset()
+   obs, reward, terminated, truncated, info = env.step({
+       "players": {
+           "env_id": info["players"]["env_id"],
+           "action": np.full(info["players"]["env_id"].shape, 2, dtype=np.int32),
+       }
+   })
+
+
+Available Tasks
+---------------
+
+Task IDs follow the pinned upstream registry. Note that upstream names
+``MarlGrid-1AgentCluttered15x15-v0`` as ``15x15`` even though that pinned
+registry config uses ``grid_size=11``.
+
+* ``MarlGrid-1AgentCluttered15x15-v0``
+* ``MarlGrid-3AgentCluttered11x11-v0``
+* ``MarlGrid-3AgentCluttered15x15-v0``
+* ``MarlGrid-2AgentEmpty9x9-v0``
+* ``MarlGrid-3AgentEmpty9x9-v0``
+* ``MarlGrid-4AgentEmpty9x9-v0``
+* ``Goalcycle-demo-solo-v0``

docs/index.rst

@@ -94,6 +94,7 @@ stable version through `envpool.readthedocs.io/en/stable/
    env/gfootball
    env/highway
    env/jumanji
+   env/marlgrid
    env/minigrid
    env/gymnasium_robotics
    env/metaworld

docs/spelling_wordlist.txt

@@ -109,3 +109,12 @@ rollouts
 filesystem
 basenames
 vendored
+rasterization
+MarlGrid
+kandouss
+multi-agent
+partial-view
+Goalcycle
+GoalCycle
+gridworld
+Ndousse

envpool/BUILD

@@ -39,6 +39,7 @@ py_library(
         "//envpool/gfootball:gfootball_registration",
         "//envpool/highway:highway_registration",
         "//envpool/jumanji:jumanji_registration",
+        "//envpool/marlgrid:marlgrid_registration",
         "//envpool/minigrid:minigrid_registration",
         "//envpool/mujoco:metaworld_registration",
         "//envpool/mujoco:mujoco_dmc_registration",
@@ -67,6 +68,7 @@ py_library(
         "//envpool/gfootball",
         "//envpool/highway",
         "//envpool/jumanji",
+        "//envpool/marlgrid",
         "//envpool/minigrid",
         "//envpool/mujoco",
         "//envpool/pgx",

envpool/entry.py

@@ -19,6 +19,7 @@
 import envpool.gfootball.registration  # noqa: F401
 import envpool.highway.registration  # noqa: F401
 import envpool.jumanji.registration  # noqa: F401
+import envpool.marlgrid.registration  # noqa: F401
 import envpool.minigrid.registration  # noqa: F401
 import envpool.mujoco.dmc.registration  # noqa: F401
 import envpool.mujoco.gym.registration  # noqa: F401

envpool/make_test.py

@@ -69,6 +69,10 @@
     "envpool.gfootball": ("gfootball/academy_empty_goal_close-v1",),
     "envpool.highway": ("HighwayFast-v0",),
     "envpool.jumanji": ("Game2048-v1",),
+    "envpool.marlgrid": (
+        "MarlGrid-2AgentEmpty9x9-v0",
+        "Goalcycle-demo-solo-v0",
+    ),
     "envpool.minigrid": ("MiniGrid-DoorKey-8x8-v0",),
     "envpool.mujoco.dmc": ("WalkerWalk-v1",),
     "envpool.mujoco.gym": ("Ant-v5",),

envpool/marlgrid/BUILD

@@ -0,0 +1,68 @@
+# Copyright 2026 Garena Online Private Limited
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+load("@pybind11_bazel//:build_defs.bzl", "pybind_extension")
+load("@rules_cc//cc:defs.bzl", "cc_library")
+load("@rules_python//python:defs.bzl", "py_library", "py_test")
+load("//envpool:requirements.bzl", "requirement")
+
+package(default_visibility = ["//visibility:public"])
+
+cc_library(
+    name = "marlgrid_env",
+    hdrs = ["marlgrid.h"],
+    deps = [
+        "//envpool/core:async_envpool",
+        "//envpool/core:logging",
+    ],
+)
+
+pybind_extension(
+    name = "marlgrid_envpool",
+    srcs = ["marlgrid.cc"],
+    deps = [
+        ":marlgrid_env",
+        "//envpool/core:py_envpool",
+    ],
+)
+
+py_library(
+    name = "marlgrid",
+    srcs = ["__init__.py"],
+    data = [":marlgrid_envpool"],
+    imports = ["../.."],
+    deps = ["//envpool/python:api"],
+)
+
+py_library(
+    name = "marlgrid_registration",
+    srcs = ["registration.py"],
+    imports = ["../.."],
+    deps = ["//envpool:registration"],
+)
+
+py_test(
+    name = "marlgrid_test",
+    size = "large",
+    srcs = ["marlgrid_test.py"],
+    data = ["@marlgrid//:source"],
+    imports = ["../.."],
+    deps = [
+        ":marlgrid",
+        ":marlgrid_registration",
+        requirement("absl-py"),
+        requirement("gymnasium"),
+        requirement("numpy"),
+    ],
+)

envpool/marlgrid/__init__.py

@@ -0,0 +1,30 @@
+# Copyright 2026 Garena Online Private Limited
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""MarlGrid env in EnvPool."""
+
+from envpool.python.api import py_env
+
+from .marlgrid_envpool import _MarlGridEnvPool, _MarlGridEnvSpec
+
+(
+    MarlGridEnvSpec,
+    MarlGridDMEnvPool,
+    MarlGridGymnasiumEnvPool,
+) = py_env(_MarlGridEnvSpec, _MarlGridEnvPool)
+
+__all__ = [
+    "MarlGridEnvSpec",
+    "MarlGridDMEnvPool",
+    "MarlGridGymnasiumEnvPool",
+]

envpool/marlgrid/marlgrid.cc

@@ -0,0 +1,24 @@
+// Copyright 2026 Garena Online Private Limited
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+#include "envpool/marlgrid/marlgrid.h"
+
+#include "envpool/core/py_envpool.h"
+
+using MarlGridEnvSpec = PyEnvSpec<marlgrid::MarlGridEnvSpec>;
+using MarlGridEnvPool = PyEnvPool<marlgrid::MarlGridEnvPool>;
+
+PYBIND11_MODULE(marlgrid_envpool, m) {
+  REGISTER(m, MarlGridEnvSpec, MarlGridEnvPool)
+}