Why is feature scaling critical from a linear algebra perspective?

Feature scaling improves conditioning of data matrices.

Without scaling, features with large magnitude dominate inner products and gradients. Scaling ensures isotropic geometry, improving numerical stability and optimization convergence.