What is temporal attention in video transformers?

Temporal attention models dependencies across frames in video sequences.

Temporal attention extends self-attention across time dimension, allowing tokens from different frames to interact. This captures motion dynamics and long-range temporal dependencies. It is essential in video transformers where understanding actions requires reasoning across multiple frames rather than single images.