What is reinforcement learning from human feedback (RLHF)?

RLHF aligns models with human preferences using reinforcement learning.

A reward model is trained from human rankings, then used to fine-tune a policy model via PPO or similar algorithms.