What is AdamW optimizer and how is it different from Adam?

AdamW decouples weight decay from gradient-based updates, improving generalization.

Adam applies L2 regularization incorrectly by mixing it into gradient updates. AdamW fixes this by applying weight decay directly to parameters, leading to better convergence and generalization, especially in Transformers.