How do transformer models represent and propagate information across layers?

Information flows through alternating attention and feed-forward transformations, refining representations layer by layer.

Transformers build hierarchical representations by iteratively mixing token-level context (via self-attention) and applying nonlinear transformations (via FFNs). Early layers encode lexical and syntactic patterns, while deeper layers encode semantic and task-specific abstractions. Residual connections preserve identity flow, enabling stable gradient propagation across deep stacks.