What is attention mechanism in neural networks?

Attention allows models to focus on relevant parts of input data dynamically.

It computes weighted importance scores for inputs using query-key-value structure, enabling context-aware representations.