An analytical estimate, not a measured benchmark. Use it to compare compute and parallelism trends; the defaults are not calibrated on hardware, and peak memory and output quality are not checked.

Estimated time breakdown for this setting

Changing only the GPU count: scaling

Each row is the end-to-end time of the same setting on a different number of GPUs (shared scale). On the right are the total and the scaling efficiency (1-GPU time ÷ (G × G-GPU time)). Below the dashed line, the sequence-parallel group spans more than one 8-GPU node.

Changing only the step count: how the shares shift

Each row shows the estimated time shares of the same setting at a different step count (100% stacked), with the total on the right. Only the number of calls changes; this does not mean the settings give the same quality, and it is not the same as few-step distillation.

Data table

Model assumptions

• tokens = latent frames × (H / s / 2) × (W / s / 2), frames = length × fps + 1, latent frames = 1 + (frames − 1) / 4. Length is the interval between the first and last frame; s is the VAE spatial compression (8 for Wan2.1, 16 for Wan2.2 5B), and the patch is 1×2×2.
• FLOPs per forward pass ≈ 2·L·n·(4d² + 2d·d_ff) (self-attention projections and FFN) + 4·L·n²·d·k (self-attention, k = fraction kept), matching the formulas in the post. Cross-attention, modulation, norms and the like are ignored; this is not the full model FLOPs.
• DiT compute time = FLOPs / (SP degree × peak × MFU). All compute shares one fixed MFU; lower efficiency on small blocks and per-kernel throughput differences are not modeled. This is a simplified trend model, and the sign of its error is not guaranteed.
• Sparse attention scales ideally with the fraction kept, ignoring pattern selection, indexing and irregular memory access; communication still sends the full K/V. Quality and training cost are not modeled.
• CFG parallel: with CFG on and at least 2 GPUs, the two forward passes run on two GPU groups, each with SP degree G / 2; the communication to combine them is ignored.
• 8 GPUs per node. An SP group of up to 8 GPUs uses intra-node bandwidth; a larger one is limited by inter-node bandwidth.
• Ulysses: counted as four tensors (Q, K, V, O) per layer, each GPU sends 8·(n/G)·d·(u−1)/u bytes, not overlapped with compute; QKV may be sent together, and the degree u must divide the head count. Bandwidth is the effective rate for that send volume.
• Ring: R−1 steps per layer, each sending 4·(n/G)·d bytes of K/V, overlapped with that step's block attention; only the part that exceeds compute time is counted.
• USP: the Ulysses degree is the largest value up to 8 that divides both the head count and the SP degree; the rest is Ring.
• VAE time scales linearly with pixel count and by default does not parallelize across GPUs; with VAE parallelism on, ideal linear speedup is assumed. The text encoder time is fixed.
• Collective launch, sequence padding, weight loading, output transfer and video encoding are ignored. The Ulysses feasibility check only tests head divisibility, not memory, the generation settings the model supports, or kernel limits.
• Layer counts, hidden sizes and head counts in the presets come from public configs; the default MFU, bandwidth and VAE values are placeholders, not calibrated against measurements.