What is attention rollout and how is it used for interpretability in Vision Transformers?

Attention rollout aggregates attention maps across layers to trace information flow in ViTs.

In Vision Transformers, each layer produces attention matrices showing token interactions. However, raw attention is hard to interpret. Attention rollout multiplies attention matrices across layers to compute effective global influence of input patches on outputs. This helps visualize which image regions contribute most to final predictions, improving model explainability.