What is ChatGPT and how does it fundamentally work?

ChatGPT is a transformer-based large language model that generates text by predicting the next token based on context.

ChatGPT is built on the Transformer architecture, which uses self-attention mechanisms to understand relationships between tokens in a sequence. It is trained on massive datasets using unsupervised learning (next-token prediction) and then fine-tuned using Reinforcement Learning from Human Feedback (RLHF). The model does not “understand” language like humans; instead, it learns statistical patterns and probabilities of token sequences.