What is positional encoding and why is it necessary in Vision Transformers?

Positional encoding injects spatial information into transformer inputs.

Transformers are permutation-invariant, meaning they cannot inherently understand spatial structure. Positional encoding adds location information to patch embeddings using sinusoidal functions or learnable vectors, enabling the model to understand relative and absolute positions of image patches.