What are emergent abilities in large language models?

Emergent abilities are capabilities that appear only after models reach a certain scale.

As model size increases, new capabilities such as in-context learning, arithmetic reasoning, and instruction following emerge non-linearly. These are not explicitly programmed but arise from distributed representations and scaling effects. This challenges traditional notions of incremental performance improvement.