How do you design an end-to-end LLM observability and tracing system?

An end-to-end LLM observability system traces prompts, retrieval, model inference, and outputs with structured logs, metrics, and distributed tracing.

LLM observability requires tracing multiple stages: user input → prompt construction → retrieval (RAG) → model inference → post-processing → final response. Each stage emits structured telemetry including latency, token usage, retrieval scores, model version, and safety flags. Unlike traditional systems, LLM observability also tracks semantic quality signals like hallucination probability and embedding drift.