The disk is filling up

Symptom: the alert Node … is low on disk space, or the node stops accepting associations.

What fills a node’s data volume:

1. Queues for a destination that is down or slow. Check the Nodes tab: a destination with a long queue and an old oldest instance. Fix the destination, or see A destination’s queue keeps growing.
2. The resend cache. Delivered instances are kept for Resend cache (hours), up to Resend cache limit (GB per node). Lower the limit if it does not fit the volume.
3. Dead letters kept for Dead-letter retention (days): re-queue or purge them.
4. The quarantine, when on, for Quarantine (days).
5. Studies held by routes that wait for whole studies, or for reconciliation.

What the node does

Below Minimum free disk (10 GB by default) a node stops listening, as if drained: a load balancer sends new associations to other nodes, and the node keeps delivering its queues, which frees space. It listens again once it has 25% more than the minimum (at least 2 GB more). Below half the minimum, instances arriving on associations already open are refused with out of resources (A700), which senders retry.

Size the data volume as in Plan and size.