What is model caching in inference systems?

Model caching stores frequent predictions to reduce inference latency.

Caching avoids recomputation for repeated inputs. It is useful in recommendation systems and NLP APIs. Cache invalidation strategies are critical for correctness.