Question: which parts of the Genesis World 1.0 blog are in this open-source release? #2871
Replies: 2 comments 1 reply
-
|
Your analysis is correct. I will see what we can do for 4 and 8. Regarding 6 and 7, this may change in the future but we are nowhere near open-sourcing the reconstruction pipeline and Nyx for now. For the other points, there is no plan to make them open-source at the moment I'm afraid. |
Beta Was this translation helpful? Give feedback.
-
|
I just read about the evaluation harness and I saw this comment and wanted to add here that a simple setup can cover a lot of ground. For example:
Literally in minutes you can/should be able to setup something that can spin, let's say 20 genesis sessions. There must be a better way to do this but at least you can pretty much x20 (in this case) your evaluation pipeline kinda for free (assuming you have the memory to do that which you should cause you are most likely using a GPU). Each session could be testing different trained models to compare them (loading local models from your past training for example) or even the same mode under different environments or even parameters of one environment. This is only a starting kind of setup and I will leave the rest to you... Note: The LLM/Agentic component can be thrown out once you have fine-tuned your pipeline. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi team,
First off — really impressive write-up. Framing simulation as an evaluation engine rather than just a data generator is a refreshing take, and the physics depth (the External Articulation Constraint on top of libuipc, the barrier-free / AL-IPC elastodynamics, the three interchangeable couplers) is a great read. Reading the post against the open repos, a lot of it lines up and is reviewable: the 8 solvers + 3 couplers (
genesis/engine/simulator.py,genesis/engine/couplers/), the libuipc IPC path (genesis/engine/couplers/ipc_coupler/), Quadrants, the 3DGS loading example (genesis-nyx/examples/05_gaussian_splat.py), and the rsl_rl + behavior-cloning examples. Thanks for putting that much out.I'm trying to reproduce and build on some of what the post describes, and a few things I genuinely couldn't locate in the current snapshot (genesis-world
6007f263, genesis-nyx, quadrants). I'm not assuming anything is withheld on purpose — I just want to understand what's in scope for the open release vs. internal, so I don't chase code that was never meant to ship. Each item below says exactly what I did and didn't find:Robot embodiments. The post features Wuji, Sharpa, Genesis hand, Pika gripper, and Tianji. I went through
genesis/assets/urdf/andgenesis/assets/xml/and grepped the repos (case-insensitive, plus the HuggingFace download paths intests/) and couldn't find URDF/MJCF/USD for any of the five. The bundled robots I do see are go2, anymal_c, kuka_iiwa, panda/franka, shadow_hand, ur5e, and drones (G1 only appears as a HF download intests/test_rigid_benchmarks.py). Are the five planned for release, or internal-only for now?Evaluation harness. The systematic closed-loop evaluation (hundreds of tasks × hundreds of episodes, the ~10 perturbation axes) is the part I was most interested in reproducing, but I couldn't find it. The only
*_eval.pyfiles in the repo areexamples/manipulation/grasp_eval.py,examples/locomotion/go2_eval.py, andexamples/drone/hover_eval.py— each loads a single checkpoint and rolls one policy out in a viewer for visualization/video. There's no task/episode orchestration, no perturbation sweep, and no metric aggregation (grep forperturbation/robustnessin an evaluation sense returned nothing). Is the harness itself planned for open-source, or is it internal tooling for now?Closed-loop / VLA policy evaluation. To be precise about what's in the repo today: the closest thing to a vision policy is the two-stage grasp example (
examples/manipulation/grasp_train.py --stage=bc), which is offline behavior cloning of a state-based RL teacher on stereo-RGB inputs (GraspEnv.get_stereo_rgb_images()) — not a closed-loop evaluation of a large/VLA policy. I couldn't find a VLA policy interface, a closed-loop eval driver, or any LeRobot/foundation-policy integration. Is the closed-loop evaluation pipeline (and the evaluated models) something you plan to share even as a minimal reference, or is that internal?Sim-to-real metrics. The 0.8996 Pearson correlation, the MMRV (0.0166), and the FID reality-gap numbers are great anchors. I couldn't find the code that computes any of them (grep for
MMRV/FID/pearson/correlationreturned nothing in the repo). Are the scripts behind these metrics — and ideally any releasable slice of the real-vs-sim data and the 14-task setup — something the community could access?Task / scene generation. The programmatic pipeline for scene layout, asset selection, environment code, and success metrics sounds very useful, but I couldn't find it (no
scene_gen/task_gen/ reward-spec module; what exists is manual reward dicts in the example envs andTerrain.randomize). Is that pipeline planned for release?Photogrammetry / reconstruction. The capture-to-3D-map pipeline (in-house iOS app, mesh + Gaussian-splat training) — I couldn't find any of it (no COLMAP/photogrammetry/reconstruction code beyond particle surface meshing). Will any of this be open, or is the plan more to release datasets?
Nyx core + the 4 ms number. The genesis-nyx repo ships docs, examples, and a prebuilt wheel, but I couldn't find the renderer's C++/CUDA source (the CI hints at an internal source repo). Is Nyx intended to stay binary-only, or is a source release on the roadmap? Separately, the repo has no benchmark for the "noise-free 1080p in 4 ms" figure — is there a script + GPU/scene/spp behind that number you could share?
Performance numbers. For the 4.6× Quadrants runtime speedup and the 103× barrier-free vs. traditional IPC figure — could you point to the baseline/config or a benchmark script so they're reproducible? (The 10× startup speedup I could trace to the fastcache README's 7.2s→0.3s, so that one is clear.)
No rush on any of this, and completely understand if some of it stays internal for now — I just want to set expectations correctly before building on it. Thanks again for the work and for open-sourcing as much as you have.
Beta Was this translation helpful? Give feedback.
All reactions