What is an activation function in Deep Learning?

Activation functions introduce nonlinearity into neural networks, enabling them to learn complex patterns.

Without activation functions, neural networks would behave like linear regression regardless of depth. Activation functions transform neuron outputs into nonlinear representations, enabling learning of complex relationships. Common functions include ReLU, Sigmoid, Tanh, and Softmax. ReLU is widely used because it reduces vanishing gradients and improves training speed. Different activations are chosen depending on task type and network architecture.