What is the purpose of activation functions in supervised learning models?

Activation functions introduce non-linearity into models.

Without activation functions, neural networks behave like linear models regardless of depth. Activation functions like ReLU, sigmoid, and tanh allow networks to learn complex nonlinear patterns. They determine whether neurons should activate and propagate signals forward.