What is the role of activation scaling in residual networks?

Activation scaling stabilizes signal magnitude in deep residual networks.

Without scaling, residual additions can amplify or diminish signal variance across layers. Proper initialization and normalization ensure stable gradient flow through deep skip connections.