What is Masked Autoencoders (MAE) in Vision Transformers and why does masking work so well?

MAE pretrains Vision Transformers by reconstructing missing image patches from heavily masked inputs.

Masked Autoencoders (MAE) apply a high masking ratio (often 75%) to image patches and train a Vision Transformer encoder-decoder to reconstruct missing patches. The encoder only processes visible tokens, while a lightweight decoder reconstructs the full image. This forces the model to learn global semantic structure instead of local pixel memorization. The asymmetry (heavy encoder, light decoder) makes training efficient and scalable.