feat(pipe): Add layout generation by text desc pipe and release v0.1.3. (#33)

* feat(pipe): Add layout generation pipe.

* tag v0.1.3

* chore: update assets

* update

---------

Co-authored-by: xinjie.wang <xinjie.wang@gpu-4090-dev015.hogpu.cc>

Author: HochCC

CHANGELOG.md

@@ -4,8 +4,18 @@ All notable changes to this project will be documented in this file.
 
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0).
 
+## [0.1.3] - 2025-08-19
+### Feature
+- Release `layout-cli` 🏞️ Layout(Interactive 3D Worlds) Generation, generate interactive 3D scene from task description.
+
+
+## [0.1.2] - 2025-07-31
+### Feature
+- Release `scene3d-cli` 🌍 3D Scene Generation, generate a color mesh and 3DGS scene from text description.
+- Release docker file and docker image.
+
 
-## [0.1.1] - 2025-07-xx
+## [0.1.1] - 2025-07-11
 ### Feature
 - Added intelligent quality checkers and auto-retry pipeline for `image-to-3d` and `text-to-3d`.
 - Added unit tests for quality checkers.

README.md

@@ -30,7 +30,7 @@
 ```sh
 git clone https://github.com/HorizonRobotics/EmbodiedGen.git
 cd EmbodiedGen
-git checkout v0.1.2
+git checkout v0.1.3
 git submodule update --init --recursive --progress
 conda create -n embodiedgen python=3.10.13 -y # recommended to use a new env.
 conda activate embodiedgen
@@ -194,8 +194,6 @@ CUDA_VISIBLE_DEVICES=0 scene3d-cli \
 
 ### 💬 Generate Layout from task description
 
-🚧 *Coming Soon*
-
 <table>
   <tr>
     <td><img src="apps/assets/layout1.gif" alt="layout1" width="360"/></td>
@@ -207,6 +205,47 @@ CUDA_VISIBLE_DEVICES=0 scene3d-cli \
   </tr>
 </table>
 
+Text-to-image model based on SD3.5 Medium, usage requires agreement to the [model license](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium). All models auto-downloaded at the first run.
+
+You can generate any desired room as background using `scene3d-cli`. As each scene takes approximately 30 minutes to generate, we recommend pre-generating them for efficiency and adding them to `outputs/bg_scenes/scene_list.txt`.
+
+We provided some sample background assets created with `scene3d-cli`. Download them(~4G) using `hf download xinjjj/scene3d-bg --repo-type dataset --local-dir outputs`.
+
+Generating one interactive 3D scene from task description with `layout-cli` takes approximately 30 minutes.
+```sh
+layout-cli --task_descs "Place the pen in the mug on the desk" "Put the fruit on the table on the plate" \
+--bg_list "outputs/bg_scenes/scene_list.txt" --output_root "outputs/layouts_gen" --insert_robot
+```
+
+<table>
+  <tr>
+    <td><img src="apps/assets/Iscene_demo1.gif" alt="Iscene_demo1" width="300"/></td>
+    <td><img src="apps/assets/Iscene_demo2.gif" alt="Iscene_demo2" width="450"/></td>
+  </tr>
+</table>
+
+Run multiple tasks defined in `task_list.txt` in the backend.
+Remove `--insert_robot` if you don't consider the robot pose in layout generation.
+```sh
+CUDA_VISIBLE_DEVICES=0 nohup layout-cli \
+--task_descs "apps/assets/example_layout/task_list.txt" \
+--bg_list "outputs/bg_scenes/scene_list.txt" \
+--output_root "outputs/layouts_gens" --insert_robot > layouts_gens.log &
+```
+
+Using `compose_layout.py`, you can recompose the layout of the generated interactive 3D scenes. (Support for texture editing and augmentation will be added later.)
+```sh
+python embodied_gen/scripts/compose_layout.py \
+--layout_path "outputs/layouts_gens/task_0000/layout.json" \
+--output_dir "outputs/layouts_gens/task_0000/recompose" --insert_robot
+```
+
+We provide `sim-cli`, that allows users to easily load generated layouts into an interactive 3D simulation using the SAPIEN engine (will support for more simulators in future updates).
+
+```sh
+sim-cli --layout_path "outputs/layouts_gens/task_0000/recompose/layout.json" \
+--output_dir "outputs/layouts_gens/task_0000/recompose/sapien_render" --robot_name "franka"
+```
 
 ### 🖼️ Real-to-Sim Digital Twin
 

apps/assets/Iscene_demo1.gif

@@ -0,0 +1,18 @@
+Place the pen in the mug on the desk
+Get the pen from the table and put it inside the cup
+Put the fruit on the table on the plate
+Put the apples on the table on the plate
+Put the spoon on the plate in the table
+Put the green lid on the desk
+Put the pen on the tray
+Move the plate forward
+Move the lamp to the left
+Pick up the cup and put on the middle of the table
+Close the laptop
+Close the red jar with the lid on the table
+Pick the cloth and shove it in the box
+Pick up the marker from the table and put it in the bowl
+Pick up the charger and move it slightly to the left
+Move the jar to the left side of the desk
+Pick the rubik's cube on the top of the desk
+Move the mug to the right

apps/assets/Iscene_demo2.gif

@@ -189,7 +189,7 @@ def build_covariance_from_scaling_rotation(
 lighting_css = """
 <style>
 #lighter_mesh canvas {
-    filter: brightness(1.8) !important;
+    filter: brightness(1.9) !important;
 }
 </style>
 """

apps/assets/example_layout/task_list.txt

@@ -51,17 +51,15 @@ class RenderResult:
 
     def __post_init__(self):
         if isinstance(self.rgb, torch.Tensor):
-            rgb = self.rgb.detach().cpu().numpy()
-            rgb = (rgb * 255).astype(np.uint8)
-            self.rgb = cv2.cvtColor(rgb, cv2.COLOR_BGR2RGB)
+            rgb = (self.rgb * 255).to(torch.uint8)
+            self.rgb = rgb.cpu().numpy()[..., ::-1]
         if isinstance(self.depth, torch.Tensor):
-            self.depth = self.depth.detach().cpu().numpy()
+            self.depth = self.depth.cpu().numpy()
         if isinstance(self.opacity, torch.Tensor):
-            opacity = self.opacity.detach().cpu().numpy()
-            opacity = (opacity * 255).astype(np.uint8)
-            self.opacity = cv2.cvtColor(opacity, cv2.COLOR_GRAY2RGB)
+            opacity = (self.opacity * 255).to(torch.uint8)
+            self.opacity = opacity.cpu().numpy()
             mask = np.where(self.opacity > self.mask_threshold, 255, 0)
-            self.mask = mask[..., 0:1].astype(np.uint8)
+            self.mask = mask.astype(np.uint8)
             self.rgba = np.concatenate([self.rgb, self.mask], axis=-1)