How do you design observability for ML systems beyond standard application monitoring?

Layer four signal types: infrastructure health, model behaviour (prediction distributions), data quality (input drift and nulls), and business outcomes — with tracing that links a prediction to the features, model version and downstream decision.

Application monitoring answers "is the service up". ML systems fail while perfectly up, so the observability model needs additional layers.

Layer 1 — Infrastructure. Invocations, ModelLatency, OverheadLatency, Invocation4XX/5XXErrors, instance CPU/GPU/memory. Necessary and insufficient.

Layer 2 — Model behaviour. The distribution of what the model outputs:…