What is deformable attention in modern transformer architectures?

Deformable attention attends to a small set of learned sampling points instead of all tokens.

Standard self-attention computes interactions between all tokens, which is expensive. Deformable attention selects a small number of key sampling locations per query and computes attention only over them. This significantly reduces complexity while preserving performance, especially in high-resolution vision tasks like detection and segmentation.