How does retrieval-augmented generation (RAG) integrate with ChatGPT architecture?

RAG combines external knowledge retrieval with LLM generation to improve factual accuracy.

Retrieval-Augmented Generation (RAG) enhances ChatGPT by connecting it to external knowledge sources like vector databases or search engines. When a query is received, relevant documents are retrieved and appended to the prompt context before generation.

This improves factual grounding and reduces hallucinations. Architecturally, RAG systems include embedding models, vector search indexes, rerankers, and the main LLM generator. The tradeoff is added latency due to retrieval steps.