RuntimeError: Failed to load graph_do_while condition kernel fatbin (CUDA error 200). This SM (120) may not be included in the fatbin — regenerate with scripts/build_condition_kernel_fatbin.py
scene.step()
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/utils/misc.py", line 142, in wrapper
return method(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/engine/scene.py", line 1047, in step
self._sim.step()
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/engine/simulator.py", line 282, in step
self.rigid_solver.substep(self.cur_substep_local)
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/engine/solvers/rigid/rigid_solver.py", line 1056, in substep
self._func_constraint_force()
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/engine/solvers/rigid/rigid_solver.py", line 1147, in _func_constraint_force
self.constraint_solver.resolve(self.entities_info, self._rigid_global_info)
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/engine/solvers/rigid/constraint/solver.py", line 259, in resolve
func_solve_body(
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/_perf_dispatch.py", line 330, in __call__
res = dispatch_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/_perf_dispatch.py", line 60, in __call__
return self.__wrapped__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/genesis/engine/solvers/rigid/constraint/solver_breakdown.py", line 1054, in func_solve_decomposed
_kernel_solve_graph(
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/_quadrants_callable.py", line 96, in __call__
return self.wrapper.__call__(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/kernel_impl.py", line 147, in wrapped_func
return primal(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/shell.py", line 29, in new_call
ret = old_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/kernel.py", line 739, in __call__
ret = self.launch_kernel(key, kernel_cpp, compiled_kernel_data, *py_args, qd_stream=qd_stream)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/kernel.py", line 591, in launch_kernel
raise e from None
File "/home/admin/miniconda3/envs/genesis/lib/python3.12/site-packages/quadrants/lang/kernel.py", line 583, in launch_kernel
prog.launch_kernel(compiled_kernel_data, launch_ctx)
RuntimeError: [graph_manager.cpp:ensure_condition_kernel_loaded@160] Failed to load graph_do_while condition kernel fatbin (CUDA error 200). This SM (120) may not be included in the fatbin — regenerate with scripts/build_condition_kernel_fatbin.py
[Genesis] [17:26:41] [ERROR] RuntimeError: [graph_manager.cpp:ensure_condition_kernel_loaded@160] Failed to load graph_do_while condition kernel fatbin (CUDA error 200). This SM (120) may not be included in the fatbin — regenerate with scripts/build_condition_kernel_fatbin.py
Bug Description
After upgrading
genesis-worldfrom0.4.5to1.1.1via pip, I encountered a CUDA compatibility issue:RuntimeError: Failed to load graph_do_while condition kernel fatbin (CUDA error 200). This SM (120) may not be included in the fatbin — regenerate with scripts/build_condition_kernel_fatbin.py
The same environment works correctly with the
0.4.5release, so this seems like a compatibility regression in1.1.1. Thank you for your help.Steps to Reproduce
above
Expected Behavior
fix this bug
Screenshots/Videos
No response
Relevant log output
Environment
Release version or Commit ID
v1.1.2, v1.1.1 and v1.1.0
Additional Context
No response