@@ -17,86 +17,54 @@ Get Started
1717===========
1818
1919Welcome to FlashDreams! This page will guide you from a fresh checkout
20- of the repository to running NVIDIA OmniDreams, the interactive
21- HDMap-conditioned driving world-model demo built on FlashDreams.
20+ of the repository to a running model. It uses :doc: `NVIDIA OmniDreams
21+ </models/omnidreams>`, the interactive driving world model, as the
22+ example; the :doc: `model gallery </models/index >` lists the run command
23+ for every other model.
2224
2325Install
2426-------
2527
26- FlashDreams uses the ``uv `` python package manager. Installation
27- instructions for ``uv `` are available in the `Astral documentation
28- <https://docs.astral.sh/uv/getting-started/installation/> `_.
29-
30- With ``uv `` installed, clone the repository and synchronize the OmniDreams
31- workspace environment:
28+ FlashDreams uses the ``uv `` Python package manager (`installation
29+ instructions <https://docs.astral.sh/uv/getting-started/installation/> `_).
30+ With ``uv `` installed, clone the repository and synchronize the
31+ OmniDreams workspace:
3232
3333.. code-block :: bash
3434
3535 git clone https://github.com/NVIDIA/flashdreams.git
3636 cd flashdreams
3737 uv sync --package flashdreams-omnidreams --extra interactive-drive
3838
39- This installs the FlashDreams core package, the OmniDreams integration,
40- and the ``interactive-drive `` demo dependencies. The unified runner CLI is
41- also available through the OmniDreams workspace member:
42-
43- .. code-block :: bash
44-
45- uv run --package flashdreams-omnidreams flashdreams-run --help
46-
47- If you are developing FlashDreams itself or want every shipped runner in
48- one environment, use ``uv sync --extra dev --extra runners `` instead.
49-
50- Hugging Face authentication
51- ---------------------------
52-
53- OmniDreams downloads demo scenes and checkpoints from `Hugging Face
54- <https://huggingface.co/> `_ the first time you run it. Before launching
55- the demo, use a Hugging Face account with access to:
56-
57- - `nvidia/omni-dreams-models
58- <https://huggingface.co/nvidia/omni-dreams-models> `_: checkpoints.
59- - `nvidia/omni-dreams-scenes
60- <https://huggingface.co/datasets/nvidia/omni-dreams-scenes> `_: demo
61- scene data.
62- - `nvidia/omni-dreams-samples
63- <https://huggingface.co/datasets/nvidia/omni-dreams-samples> `_ if you
64- plan to run the scripted example-data command later on this page.
65-
66- Then create a read token from `Hugging Face token settings
67- <https://huggingface.co/settings/tokens> `_. The token must have read
68- access to ``nvidia/omni-dreams-models `` and
69- ``nvidia/omni-dreams-scenes ``. If you use a fine-grained token, include
70- both repositories, plus ``nvidia/omni-dreams-samples `` if you plan to run
71- the scripted example-data command. Export the token in the same shell
72- where you run ``omnidreams-prepare ``, ``interactive-drive ``, or
73- ``flashdreams-run ``:
74-
75- .. code-block :: bash
76-
77- export HF_TOKEN=hf_...
78-
79- If the default cache under ``~/.cache/huggingface `` is on a small disk,
80- point ``HF_HOME `` at a larger volume before the first download:
39+ Most runs need a Hugging Face token. For OmniDreams, use a token with
40+ read access to `nvidia/omni-dreams-models
41+ <https://huggingface.co/nvidia/omni-dreams-models> `_ and
42+ `nvidia/omni-dreams-scenes
43+ <https://huggingface.co/datasets/nvidia/omni-dreams-scenes> `_:
8144
8245.. code-block :: bash
8346
84- export HF_HOME=/path/to/large/huggingface-cache
47+ export HF_TOKEN= < your-hf-token >
8548
86- For more environment and container details, see the project
87- `README <https://github.com/NVIDIA/flashdreams/blob/main/README.md >`_.
49+ For container, caching, and other environment details, see the project
50+ `README <https://github.com/NVIDIA/flashdreams/blob/main/README.md >`_ and
51+ :doc: `/troubleshooting `.
8852
89- Run OmniDreams
90- --------------
53+ Run your first model
54+ --------------------
9155
92- Start with the :doc: `OmniDreams </models/omnidreams >` interactive driving
93- demo. The command below runs the world model in one process and streams
94- the generated camera view to a browser:
56+ Launch the OmniDreams interactive driving demo. It runs the world model
57+ and streams the generated camera view to a browser:
9558
9659.. code-block :: bash
9760
9861 uv run --package flashdreams-omnidreams interactive-drive --stream-mjpeg :8080
9962
63+ Then open ``http://<server-ip>:8080/ `` in a browser on the same network
64+ (use ``localhost `` on the same machine). The first launch spends several
65+ minutes loading checkpoints and compiling kernels; later launches reuse
66+ the cached assets.
67+
10068On VRAM-constrained GPUs, add ``--offload-text-encoder `` to reduce peak
10169VRAM usage by about 15 GB:
10270
@@ -106,87 +74,14 @@ VRAM usage by about 15 GB:
10674 --stream-mjpeg :8080 \
10775 --offload-text-encoder
10876
109- This runs the text and first-frame encoders once per scene, frees them
110- before building the diffusion pipeline, and reuses the cached embeddings
111- across world-model resets. The trade-off is slower first loads and
112- scene/variant switches.
113-
114- Then open ``http://<server-ip>:8080/ `` in a browser on the same network.
115- Use ``localhost `` when the browser is on the same machine. The default
116- scene auto-stages on first launch when ``HF_TOKEN `` is set.
117-
118- Optionally, pre-download the default scene and model assets so access
119- errors and multi-GB downloads happen before the demo starts:
120-
121- .. code-block :: bash
122-
123- uv run --package flashdreams-omnidreams omnidreams-prepare \
124- --scene-uuid clipgt-0d404ff7-2b66-498c-b047-1ed8cded60d4
125-
126- The first launch can spend several minutes loading checkpoints,
127- compiling kernels, capturing CUDA graphs, and autotuning. Later launches
128- reuse cached assets and compiled kernels.
129-
130- Scripted OmniDreams generation
131- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
132-
133- To generate a fixed OmniDreams rollout through ``flashdreams-run ``, use
134- the single-view runner with bundled example data:
135-
136- .. code-block :: bash
137-
138- uv run --package flashdreams-omnidreams \
139- flashdreams-run \
140- omnidreams-sv-2steps-chunk2-loc6-lightvae-lighttae-perf \
141- --example-data True \
142- --example_data_uuid " 239560dc-33d1-11ef-9720-00044bcbccac" \
143- --total-blocks 20
144-
145- See :doc: `/models/omnidreams ` for local-window mode, WebRTC serving,
146- scene variants, perf-manifest setup, multi-GPU launch commands, and
147- additional example-data UUIDs.
148-
149- Other methods
150- ~~~~~~~~~~~~~
151-
152- After OmniDreams, the model gallery covers the rest of the shipped
153- FlashDreams methods and their runner slugs. For example, the offline
154- :doc: `Self-Forcing </models/self_forcing >` Wan 2.1 1.3B T2V method runs
155- with:
156-
157- .. code-block :: bash
158-
159- uv run --project integrations/self_forcing \
160- flashdreams-run self-forcing-wan2.1-t2v-1.3b-taehv \
161- --total-blocks 7
162-
163- Additional setup options
164- ------------------------
165-
166- These steps are not required for the basic OmniDreams demo, but they are
167- useful for development workflows and downstream projects.
168-
169- Use FlashDreams as a library
170- ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
171-
172- If you only need FlashDreams as a dependency in another project rather
173- than running the shipped methods, install it from PyPI instead:
174-
175- .. code-block :: bash
176-
177- pip install flashdreams
178-
179- Or track the current ``main `` branch:
180-
181- .. code-block :: bash
182-
183- pip install " git+https://github.com/NVIDIA/flashdreams.git"
77+ See :doc: `/models/omnidreams ` for the offload trade-offs, scripted
78+ generation, scene variants, WebRTC serving, and multi-GPU options.
18479
18580Where to next
18681-------------
18782
188- - :doc: `/models/index `: every shipped method with its CLI slug,
189- checkpoint source, per-method knobs, and the command to run it.
83+ - :doc: `/models/index `: every shipped model with its CLI slug and the
84+ command to run it.
19085- :doc: `/models/omnidreams `: drive a world model in real time with the
19186 ``interactive-drive `` demo.
19287- :doc: `/developer_guides/inference_pipeline_overview `: the generation
@@ -198,19 +93,3 @@ Where to next
19893- :doc: `CLI and API Reference </api/index >`: Reference docs for the
19994 ``flashdreams-run `` CLI and the FlashDreams Python API.
20095- :doc: `/troubleshooting `: common first-run failures and fixes.
201-
202- Project and support
203- -------------------
204-
205- FlashDreams is developed in the open on GitHub, and contributions are
206- welcome. If you hit a bug or have a feature request, open an issue; if
207- you have a fix or improvement, send a pull request. Browsing existing
208- issues and pull requests is also a good way to see what others are
209- working on.
210-
211- - `GitHub repository <https://github.com/NVIDIA/flashdreams >`_: source,
212- releases, and documentation.
213- - `Issues <https://github.com/NVIDIA/flashdreams/issues >`_: report bugs
214- or request features.
215- - `Pull requests <https://github.com/NVIDIA/flashdreams/pulls >`_: review
216- work in progress or contribute your own.
0 commit comments