What is weight tying in language models?

Weight tying shares embedding and output projection weights to reduce parameters.

Instead of learning separate matrices for input embeddings and output logits, both use the same weight matrix, improving generalization and reducing model size.