Skip to content

Commit 2ca4c28

Browse files
committed
[perf]: clarify samples/output_batch.output equivalence; drop hardcoded transfer size
Address review comments on hao-ai-lab#1362: - Document that `samples` is just the pinned-CPU mirror of `output_batch.output` with no intervening preprocessing, so sourcing from `output_batch.output` is the same data (Copilot). - Replace the hardcoded ~0.3-1.4 GB estimate with a description of the scaling relationship; it varies with resolution/frames/batch/dtype and would rot (Copilot). - Note the SSIM-gated (not bit-exact) equivalence inline. No behavior change.
1 parent 3ecfad0 commit 2ca4c28

1 file changed

Lines changed: 14 additions & 8 deletions

File tree

fastvideo/entrypoints/video_generator.py

Lines changed: 14 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -793,14 +793,20 @@ def execute_forward_thread():
793793
frames = None if is_latent_output else []
794794
else:
795795
# Quantize on the source device (typically CUDA) BEFORE the
796-
# device->host copy. The previous path read from the pinned-CPU
797-
# `samples` buffer whose non-blocking D->H had not completed, so
798-
# the first op here blocked on the full fp32 video transfer
799-
# (~0.3-1.4 GB) and ran a single-threaded per-frame CPU *255/cast
800-
# loop. Casting to uint8 on-device first makes the transfer 4x
801-
# smaller and moves the elementwise work onto the GPU.
802-
# clamp_() also fixes a latent overflow bug: VAE output slightly
803-
# outside [0, 1] wrapped mod 256 in the old unclamped cast.
796+
# device->host copy. `samples` above is just the pinned-CPU
797+
# mirror of `output_batch.output` (`samples.copy_(output)` or
798+
# `output.cpu()`) with no intervening preprocessing, so reading
799+
# `output_batch.output` here is the same data — but its
800+
# non-blocking D->H had not completed, so the first op that
801+
# touched `samples` blocked on the full fp32 video transfer
802+
# (which scales with resolution x frames x batch and can be
803+
# large) and then ran a single-threaded per-frame CPU
804+
# *255/cast loop. Casting to uint8 on-device first makes the
805+
# transfer 4x smaller and moves the elementwise work onto the
806+
# GPU. clamp_() also fixes a latent overflow bug: VAE output
807+
# slightly outside [0, 1] wrapped mod 256 in the old unclamped
808+
# cast. (Equivalence is SSIM-gated, not bit-exact: float->uint8
809+
# differs <=1 LSB CPU vs GPU.)
804810
src = output_batch.output
805811
vid_u8 = (src * 255).clamp_(0, 255).to(torch.uint8)
806812
vid_u8 = rearrange(vid_u8, "b c t h w -> t b c h w").cpu()

0 commit comments

Comments
 (0)