How do transformer attention layers scale with sequence length?

Attention scales quadratically with sequence length in compute and memory.

Self-attention computes pairwise token interactions, leading to O(n²) complexity. This becomes expensive for long documents, motivating sparse attention and efficient transformers.