[envpool] upgrade direct dependencies and remove stale mirrors (#334)

## Summary
- Problem: EnvPool still pinned several direct Python and native
dependencies to old versions, and `WORKSPACE` still referenced the
retired `https://ml.cs.tsinghua.edu.cn/~jiayi/envpool` mirror.
- Scope: keep the retained first-round upgrades only: MuJoCo /
dm_control versions, Python requirement floors and lockfiles, selected
direct native archive versions (`pybind11`, `glog`, `googletest`,
`zlib`), and matching docs / build-file updates. This PR intentionally
excludes the experimental second-round Bazel / toolchain / broader C++
dependency bumps.
- Outcome: the retained direct dependency set is newer and consistent
across Bazel archives, Python lockfiles, MuJoCo build outputs, and docs,
and the stale mirror URLs are removed.

This narrows the dependency refresh to the direct changes that are
easiest to review and keep, while dropping the more invasive toolchain
work.

## Technical Details
- Approach: update the direct dependency versions in `workspace0.bzl`
and pip requirement files, regenerate the Linux lockfiles, and sync
MuJoCo shared-library names plus doc references to the retained
versions.
- Code pointers:
- `envpool/workspace0.bzl`: removes the stale mirror URLs and bumps the
retained native archive versions.
- `third_party/pip_requirements/requirements-dev.txt`: updates the
dev/runtime dependency inputs that feed the lockfiles.
- `envpool/mujoco/BUILD`: updates the generated MuJoCo shared library
output to match 3.6.0.
- Notes: I temporarily explored a second round of Bazel / toolchain /
C++ dependency upgrades during debugging, but rolled those changes back
before publishing this PR.

## Test Plan
### Automated
- `brix ssh dev-0 -C -- '<targeted bazel test run on the retained
first-round dependency set>'`: earlier targeted dev validation passed
the core buffer tests plus MuJoCo / dm_control alignment tests for the
retained dependency-refresh path.
### Suggested Manual
- `brix ssh dev-0 -C -- 'cd /root/code/envpool && bazel test
//envpool/mujoco:mujoco_gym_deterministic_test
//envpool/mujoco:mujoco_gym_align_test
//envpool/mujoco:mujoco_dmc_suite_deterministic_test
//envpool/mujoco:mujoco_dmc_suite_align_test --test_output=errors'`:
revalidate the final PR branch on Linux.
- `rg -n "ml\\.cs\\.tsinghua\\.edu\\.cn/~jiayi/envpool"
envpool/workspace0.bzl`: confirm the retired mirror URL is gone.

Author: Trinkle23897

benchmark/requirements.txt

@@ -1,10 +1,10 @@
 gym[accept-rom-license]==0.23.1
 ale-py==0.7.5
-mujoco==2.2.0
+mujoco==3.6.0
 envpool==0.6.1.post1
 sample-factory==1.123.0
 mujoco_py==2.1.2.14
 tqdm
 opencv-python-headless
-dm_control==1.0.3.post1
+dm_control==1.0.38
 packaging

docs/env/box2d.rst

@@ -1,8 +1,8 @@
 Box2D
 =====
 
-We use ``box2d==2.4.1`` and ``gym==0.23.1`` as the codebase. See
-https://github.com/erincatto/box2d/tree/v2.4.1 and
+We use ``box2d==2.4.2`` and ``gym==0.23.1`` as the codebase. See
+https://github.com/erincatto/box2d/tree/v2.4.2 and
 https://github.com/openai/gym/tree/v0.23.1/gym/envs/box2d
 
 
@@ -159,6 +159,6 @@ We ran gym's environments with dependency ``box2d-py==2.3.5`` (see
 https://github.com/openai/box2d-py/tree/2.3.5) for 1000 episode and averaged
 reward as our threshold.
 
-The current implementation of envpool is based on box2d 2.4.1 version.
+The current implementation of envpool is based on box2d 2.4.2 version.
 If you want to use box2d==2.3.5 to run the test, please checkout commit
 ``4de47ebb6615052c67fdfbbe9bc3e9b1d5692f99`` and build the wheel.

docs/env/dm_control.rst

@@ -1,9 +1,9 @@
 DeepMind Control Suite
 ======================
 
-We use ``dm_control==1.0.5`` and ``mujoco==3.3.4`` as the codebase.
-See https://github.com/deepmind/dm_control/tree/1.0.5 and
-https://github.com/google-deepmind/mujoco/tree/3.3.4
+We use ``dm_control==1.0.38`` and ``mujoco==3.6.0`` as the codebase.
+See https://github.com/deepmind/dm_control/tree/1.0.38 and
+https://github.com/google-deepmind/mujoco/tree/3.6.0
 
 The ``domain_name`` and ``task_name`` for ``suite.load`` function are
 converted into ``DomainNameTaskName-v1`` in envpool, e.g.,
@@ -24,7 +24,7 @@ AcrobotSwingup-v1, AcrobotSwingupSparse-v1
 ------------------------------------------
 
 `dm_control suite acrobot source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/acrobot.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/acrobot.py>`_
 
 - Observation spec: a namedtuple with two keys: ``orientations (4)``,
   ``velocity (2)``;
@@ -37,7 +37,7 @@ BallInCupCatch-v1
 -----------------
 
 `dm_control suite ball-in-cup source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/ball_in_cup.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/ball_in_cup.py>`_
 
 - Observation spec: a namedtuple with two keys: ``position (4)`` and
   ``velocity (4)``;
@@ -50,7 +50,7 @@ CartpoleBalance-v1, CartpoleBalanceSparse-v1, CartpoleSwingup-v1, CartpoleSwingu
 --------------------------------------------------------------------------------------------------------------------------------------
 
 `dm_control suite cartpole source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/cartpole.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/cartpole.py>`_
 
 - Observation spec: a namedtuple with two keys: ``position (5 for two_poles, 7
   for three_poles, 3 for others)``, ``velocity (3 for two_poles, 4 for
@@ -64,7 +64,7 @@ CheetahRun-v1
 -------------
 
 `dm_control suite cheetah source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/cheetah.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/cheetah.py>`_
 
 - Observation spec: a namedtuple with two keys: ``position (8)`` and
   ``velocity (9)``;
@@ -77,7 +77,7 @@ FingerSpin-v1, FingerTurnEasy-v1, FingerTurnHard-v1
 ---------------------------------------------------
 
 `dm_control suite finger source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/finger.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/finger.py>`_
 
 - Observation spec: a namedtuple with five keys: ``position (4)``,
   ``velocity (3)``, ``touch (2)``, ``target_position (2)``,
@@ -97,7 +97,7 @@ FishSwim-v1, FishUpright-v1
 ---------------------------
 
 `dm_control suite fish source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/fish.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/fish.py>`_
 
 - Observation spec: a namedtuple with four keys: ``joint_angles (7)``,
   ``upright ()``, ``target (3)``, ``velocity (13)``;
@@ -115,7 +115,7 @@ HopperStand-v1, HopperHop-v1
 ----------------------------
 
 `dm_control suite hopper source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/hopper.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/hopper.py>`_
 
 - Observation spec: a namedtuple with three keys: ``position (6)``,
   ``velocity (7)``, ``touch (2)``;
@@ -128,7 +128,7 @@ ManipulatorBringBall-v1, ManipulatorBringPeg-v1, ManipulatorInsertBall-v1, Manip
 --------------------------------------------------------------------------------------------------
 
 `dm_control suite manipulator source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/manipulator.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/manipulator.py>`_
 
 - Observation spec: a namedtuple with three keys: ``arm_pos (8,2)``,
   ``arm_vel (8)``, ``touch (5)``, ``hand_pos (4)``, ``object_pos (4)``,
@@ -142,7 +142,7 @@ HumanoidStand-v1, HumanoidWalk-v1, HumanoidRun-v1, HumanoidRunPureState-v1
 --------------------------------------------------------------------------
 
 `dm_control suite humanoid source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/humanoid.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/humanoid.py>`_
 
 - Observation spec: a namedtuple with seven keys: ``joint_angles (21)``,
   ``head_height ()``, ``extremities (12)``, ``torso_vertical (3)``,
@@ -164,7 +164,7 @@ HumanoidCMUStand-v1, HumanoidCMURun-v1
 --------------------------------------
 
 `dm_control suite humanoid-CMU source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/humanoid_CMU.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/humanoid_CMU.py>`_
 
 - Observation spec: a namedtuple with six keys: ``joint_angles (56)``,
   ``head_height ()``, ``extremities (12)``, ``torso_vertical (3)``,
@@ -178,7 +178,7 @@ PendulumSwingup-v1
 ------------------
 
 `dm_control suite pendulum source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/pendulum.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/pendulum.py>`_
 
 - Observation spec: a namedtuple with three keys: ``orientations (2)``,
   ``velocity (1)``;
@@ -191,7 +191,7 @@ PointMassEasy-v1, PointMassHard-v1
 ----------------------------------
 
 `dm_control suite point-mass source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/point_mass.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/point_mass.py>`_
 
 - Observation spec: a namedtuple with three keys: ``position (2)``,
   ``velocity (2)``;
@@ -204,7 +204,7 @@ ReacherEasy-v1, ReacherHard-v1
 ------------------------------
 
 `dm_control suite reacher source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/reacher.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/reacher.py>`_
 
 - Observation spec: a namedtuple with three keys: ``position (2)``,
   ``to_target (2)`` and ``velocity (2)``;
@@ -217,7 +217,7 @@ SwimmerSwimmer6-v1, SwimmerSwimmer15-v1
 ---------------------------------------
 
 `dm_control suite swimmer source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/swimmer.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/swimmer.py>`_
 
 - Observation spec: a namedtuple with three keys: ``joints (5 for swimmer6,
   14 for swimmer15)``, ``to_target (2)``, and ``body_velocities (18 for
@@ -231,7 +231,7 @@ WalkerRun-v1, WalkerStand-v1, WalkerWalk-v1
 -------------------------------------------
 
 `dm_control suite walker source code
-<https://github.com/deepmind/dm_control/blob/1.0.5/dm_control/suite/walker.py>`_
+<https://github.com/deepmind/dm_control/blob/1.0.38/dm_control/suite/walker.py>`_
 
 - Observation spec: a namedtuple with three keys: ``orientations (14)``,
   ``height ()`` and ``velocity (9)``;

docs/env/minigrid.rst

@@ -1,8 +1,8 @@
 Minigrid
 ========
 
-We use ``minigrid==2.1.0`` as the codebase.
-See https://github.com/Farama-Foundation/Minigrid/tree/v2.1.0
+We use ``minigrid==3.0.0`` as the codebase.
+See https://github.com/Farama-Foundation/Minigrid/tree/v3.0.0
 
 
 Empty

docs/env/mujoco_gym.rst

@@ -1,8 +1,8 @@
 Mujoco (gym)
 ============
 
-We use ``mujoco==3.3.4`` as the codebase.
-See https://github.com/google-deepmind/mujoco/tree/3.3.4
+We use ``mujoco==3.6.0`` as the codebase.
+See https://github.com/google-deepmind/mujoco/tree/3.6.0
 
 The implementation follows Gymnasium \*-v5 environment, see
 `reference <https://github.com/Farama-Foundation/Gymnasium/tree/v1.2.3/gymnasium/envs/mujoco>`_.

docs/env/vizdoom.rst

@@ -1,8 +1,8 @@
 ViZDoom
 =======
 
-We use ``vizdoom==1.1.13`` as the codebase. See
-https://github.com/mwydmuch/ViZDoom/tree/1.1.13
+We use ``vizdoom==1.3.0`` as the codebase. See
+https://github.com/Farama-Foundation/ViZDoom/tree/1.3.0
 
 
 Env Wrappers
@@ -51,10 +51,10 @@ Options
 * ``iwad_path (str)``: the rendering resource package path, default to
   ``vizdoom/bin/freedoom2``;
 * ``map_id (str)``: the vizdoom map id, see `setDoomMap
-  <https://github.com/mwydmuch/ViZDoom/blob/master/doc/DoomGame.md#setDoomMap>`_,
+  <https://github.com/Farama-Foundation/ViZDoom/blob/1.3.0/doc/DoomGame.md#setDoomMap>`_,
   available options are ``"map01", "map02", ...``, default to ``"map01"``;
 * ``game_args (str)``: the args string for vizdoom game, see `addGameArgs
-  <https://github.com/mwydmuch/ViZDoom/blob/master/doc/DoomGame.md#addGameArgs>`_,
+  <https://github.com/Farama-Foundation/ViZDoom/blob/1.3.0/doc/DoomGame.md#addGameArgs>`_,
   default to ``""``;
 * ``reward_config (Dict[str, Tuple[float, float]])``: how to calculate the
   reward (see below), default to ``{"FRAGCOUNT": [1, -1.5], "KILLCOUNT": [1, 0],

envpool/box2d/box2d_correctness_test.py

@@ -267,8 +267,8 @@ def solve_bipedal_walker(
     # the following number is from gym's 1000 episode mean reward
     if hardcore:  # -59.219390 ± 25.209768
       self.assertTrue(abs(mean_reward + 59) < 10, (hardcore, mean_reward))
-    else:  # 102.647320 ± 125.075071
-      self.assertTrue(abs(mean_reward - 103) < 20, (hardcore, mean_reward))
+    else:  # 145.318979 ± 126.231202 on box2d 2.4.2
+      self.assertTrue(abs(mean_reward - 145) < 20, (hardcore, mean_reward))
 
   def render_bpw(self, info: dict) -> None:
     SCALE = 30.0

envpool/core/state_buffer_queue_test.cc

@@ -114,7 +114,7 @@ TEST(StateBufferQueueTest, NumPlayers) {
     EXPECT_EQ(slice.arr[1].Shape(0), 1);
     size += num_players;
   }
-  std::vector<Array> out = queue.Wait(batch * max_num_players - size);
+  std::vector<Array> out = queue.Wait();
   EXPECT_EQ(out[0].Shape(0), size);
   EXPECT_EQ(out[1].Shape(0), batch);
 }

envpool/mujoco/BUILD

@@ -43,7 +43,7 @@ genrule(
 genrule(
     name = "gen_mujoco_so",
     srcs = ["@mujoco//:mujoco_so"],
-    outs = ["libmujoco.so.3.3.4"],
+    outs = ["libmujoco.so.3.6.0"],
     cmd = "cp $< $@",
 )
 

envpool/vizdoom/vizdoom_pretrain_test.py

@@ -15,6 +15,7 @@
 
 import multiprocessing as mp
 import os
+import queue
 import shutil
 from typing import Optional, Tuple
 
@@ -111,7 +112,7 @@ def _eval_c51_impl(
 
 
 def _eval_c51_subprocess(
-  queue: mp.Queue,
+  result_queue: mp.Queue,
   task: str,
   resume_path: str,
   cfg_path: Optional[str],
@@ -123,7 +124,7 @@ def _eval_c51_subprocess(
     cfg_path=cfg_path,
     reward_config=reward_config,
   )
-  queue.put((reward, length))
+  result_queue.put((reward, length))
 
 
 class _VizdoomPretrainTest(absltest.TestCase):
@@ -168,11 +169,11 @@ def test_d3(self) -> None:
       f.write(cfg.replace("hud = false", "hud = true"))
     try:
       ctx = mp.get_context("spawn")
-      queue: mp.Queue = ctx.Queue()
+      result_queue: mp.Queue = ctx.Queue()
       proc = ctx.Process(
         target=_eval_c51_subprocess,
         args=(
-          queue,
+          result_queue,
           "D3_battle",
           model_path,
           "d3.cfg",
@@ -182,9 +183,20 @@ def test_d3(self) -> None:
         ),
       )
       proc.start()
-      proc.join()
+      try:
+        reward, length = result_queue.get(timeout=120)
+      except queue.Empty:
+        proc.terminate()
+        proc.join(timeout=5)
+        self.fail("Timed out waiting for D3 subprocess result")
+      proc.join(timeout=30)
+      if proc.is_alive():
+        proc.terminate()
+        proc.join(timeout=5)
+        self.fail("D3 subprocess did not exit after producing a result")
       self.assertEqual(proc.exitcode, 0)
-      reward, length = queue.get_nowait()
+      result_queue.close()
+      result_queue.join_thread()
     finally:
       if os.path.exists("d3.cfg"):
         os.remove("d3.cfg")