Disen-Forcing: Disentangling Semantic Anchoring from Motion for Autoregressive Video Diffusion
Abstract
Disen-Forcing is a distillation-time method for autoregressive video diffusion models that disentangles attention-head specialization to improve long-range consistency and motion dynamics in long-horizon video generation.
Autoregressive video diffusion models (AR-VDMs) enable real-time, long-horizon video generation but degrade rapidly once the inference horizon outruns the training horizon, due to compounding exposure bias. To stabilize long rollouts, recent methods heuristically adopt frame sink, keeping initial frames as a persistent model context, inspired by attention sinks in LLMs. We revisit this design and find that the mechanism by which initial frames stabilize long-range consistency in AR-VDMs is not an LLM-style attention sink; instead, long-range consistency is anchored through a sparse subset of attention heads, while the majority of heads predominantly capture local temporal dynamics. This observation reveals an over-conditioning pathology in existing frame-sink strategies: by universally exposing every head to the pinned initial-frame context, the model becomes prone to collapsing motion toward the low-dynamic modes of the teacher during model distillation. We propose Disen-Forcing, a distillation-time plug-in for AR-VDMs that disentangles and reinforces this head specialization through disentangled context routing and motion-disentangling regularization. On long-horizon video generation benchmarks, Disen-Forcing simultaneously improves long-range consistency and motion dynamics of base AR-VDMs, a trade-off that sink-based methods fail to balance.
Community
The frame sink trick in autoregressive video diffusion models helps keep long-horizon rollouts consistent, but it also pulls motion toward low-dynamic modes. We find that long-range consistency is carried by a small subset of attention heads, while most heads handle local temporal dynamics. Disen-Forcing is a distillation-time plug-in that routes the frame sink context only to those anchoring heads and regularizes motion to be invariant to the frame sink, so long-horizon consistency and motion dynamics both improve.
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models (2026)
- LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation (2026)
- Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation (2026)
- LongForcing: Scaling Temporal Manifold Matching for Long-Horizon World Rollout (2026)
- Learning-to-Memorize: Dynamic Context Management for Long-Horizon Autoregressive Video Generation (2026)
- Clean Forcing: Drift-Resistant Autoregressive Video Diffusion with a Frozen Base (2026)
- Trajectory Forcing: Exploiting Diffusion Trajectories for Autoregressive Long Video Generation (2026)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper