How do you detect and handle model drift in LLMOps systems?

Model drift in LLMOps is detected using changes in output quality, embedding distributions, and user feedback trends, and handled via retraining, prompt updates, or model switching.

Unlike traditional ML drift (feature distribution shifts), LLM drift is more subtle and includes semantic drift, behavioral drift, and retrieval drift in RAG systems. Detection involves monitoring response embeddings, hallucination rates, user satisfaction signals, and changes in token-level behavior. Handling strategies include prompt refinement, retrieval index updates, fine-tuning, or routing to newer models.