What is RLHF and how does it redefine cost functions in LLM training?

RLHF replaces static loss functions with reward models trained from human preferences.

Reinforcement Learning from Human Feedback (RLHF) transforms the cost function into a learned reward signal. Instead of minimizing a fixed supervised loss, the model is optimized to maximize a reward model trained on human preference comparisons. This introduces a two-stage objective: supervised pretraining followed by reinforcement optimization using policy gradients. The cost function becomes dynamic, subjective, and distribution-shifted because it depends on learned human preferences rather than ground-truth labels.