ingest-worker error rate is elevated
WarningSample- What is happening
- Error rate 1.8% since 09:55 UTC, p95 940 ms.
- Impact
- ingest-worker on demo-linux-02 (UAT) · backlog replay in progress.
- Possible cause
- Host memory 82.4% and rising; coincident /var/log growth on the same server.
- Recommended next action
- Open Demo-Linux-02, inspect RSS and /var/log, then pause debug logging.
Open ingest-workerMemory pressure on Demo-Linux-02
WarningSample- What is happening
- Memory above 80% for 15 minutes (82.4% now).
- Impact
- UAT worker host demo-linux-02 · ingest-worker sharing the machine.
- Possible cause
- Worker RSS climbed after a backlog replay. Swap activity is rising.
- Recommended next action
- Inspect ingest-worker RSS on Demo-Linux-02 and reduce replay concurrency.
Open Demo-Linux-02Unexpected /var/log growth
WarningSample- What is happening
- /var/log is 22.4% of disk, 18.6% in 24 hours.
- Impact
- Storage on demo-linux-02 is the fastest-growing path in the sample fleet.
- Possible cause
- Application debug logging left enabled after yesterday’s load test.
- Recommended next action
- Disable debug logging on ingest-worker and re-check /var/log growth.
Open storage