What is token pruning in Vision Transformers and why is it useful?

Token pruning removes unimportant image tokens to reduce computation in Vision Transformers.

In ViTs, each image is split into many tokens, leading to high computational cost due to self-attention. Token pruning dynamically removes or merges less informative tokens based on attention scores or learned importance. This reduces sequence length, improving speed and memory efficiency while maintaining accuracy.