FAQ - Queue Depth Threshold Configuration
Question
How do I prevent alerts for temporary queue depth spikes?
Answer
Configure per-queue thresholds with appropriate limits for each queue's normal operating range. High-throughput queues can have higher thresholds (e.g., alert if depth >5,000) while error queues have strict thresholds (e.g., alert if depth >10). Adjust thresholds based on historical queue depth patterns and consumer capacity.
Threshold Configuration Strategy
1. Analyze Historical Patterns
Review queue depth over time (1-2 weeks) to understand:
- Peak depth during normal operations (e.g., morning batch processing)
- Temporary spikes (e.g., deployment restarts, consumer scaling events)
- Sustained growth (e.g., consumer failures, performance degradation)
2. Set Thresholds by Queue Type
Queue Type | Example Threshold | Reasoning |
---|---|---|
Critical transactional (orders, payments) | Depth >100 | Should process immediately; backlog indicates consumer failure |
High-throughput batch (ETL, reporting) | Depth >5,000 | Normal to accumulate during batch runs; alert only if excessive |
Error/dead letter | Depth >10 | Any messages indicate failures requiring immediate investigation |
Low-volume monitoring | Depth >50 | Rarely accumulates; small backlog indicates problem |
3. Configure Warning vs Critical Thresholds
Use multiple threshold levels:
- Warning (e.g., depth >500): Notify operations, investigate when convenient
- Critical (e.g., depth >2,000): Page on-call engineer, immediate action required
4. Account for Consumer Capacity
Threshold should consider:
- Consumer processing rate (messages/second)
- Expected recovery time (how long to clear backlog)
- Memory constraints (large backlogs consume RAM)
Example: Consumer processes 100 messages/second. Alert threshold set at 1,000 messages = 10 seconds of backlog = reasonable buffer before investigation required.
Configuration in Nodinite
- Navigate to Resource Configuration for RabbitMQ cluster
- Select Queue to monitor
- Configure Queue Depth Threshold:
- Warning: 500 messages
- Critical: 2,000 messages
- Configure Consumer Lag Threshold (optional):
- Warning: Lag >1 minute
- Critical: Lag >5 minutes
Next Step
Configuration Guide
Monitoring RabbitMQ Features