Correlate Heap Spikes with Application Events
This troubleshooting guide shows how to correlate a heap spike, a sudden rise in used heap memory, with application events when the Nodinite JMX Monitoring Agent changes a JVM Resource to Warning or Error because free heap dropped below the Heap Size threshold (Configuration).
Common Causes of Heap Spikes
Understanding the root cause of a heap spike helps you decide whether action is needed:
| Cause | Free heap pattern | Resolution |
|---|---|---|
| Deployment | Sharp drop, then stabilizes | ✅ Expected - allow the JVM to warm up |
| Batch Job | Low during the job, recovers after | ✅ Expected - tune the Duration |
| Traffic Surge | Gradual drop, then recovers | ✅ Expected - may need heap tuning |
| Memory Leak | Slow continuous drop, never recovers | ⚠️ Action required - investigate code |
| Garbage Collection Pressure | Stays low with only small recoveries | ⚠️ Action required - tune GC settings |
| CPU Spike | Free heap flat but the CPU threshold fires | ⚠️ Action required - analyze a thread dump |
Correlation Workflow
Use the following workflow to investigate a low free heap state:
this time?"} D -->|"Yes"| E["fa:fa-check Expected spike
Monitor and resolve"] D -->|"No"| F["fa:fa-search Check batch job logs"] F --> G{"Batch job
running?"} G -->|"Yes"| H["fa:fa-check Expected spike
Adjust Duration"] G -->|"No"| I["fa:fa-clock-rotate-left Review Resource State History"] I --> J{"Recurring and
getting worse?"} J -->|"Yes"| K["fa:fa-bug Investigate memory leak"] J -->|"No"| L["fa:fa-wrench Tune GC settings"] style A fill:#FFD525 style E fill:#93d16d style H fill:#93d16d style K fill:#c33d34
Diagram: Workflow for correlating a Heap Size Warning or Error (yellow) with application events. Green ends are expected spikes; red is a likely memory leak.
Correlation Steps
Step 1: Note the State Change Timestamp
When the JVM Resource changes to Warning or Error in Nodinite, note the exact timestamp. This is your anchor point for correlation.
Step 2: Check Deployment and Batch Schedules
Compare the timestamp against:
- Scheduled deployments and releases
- Scheduled batch jobs and data processing tasks
- Maintenance windows and JVM restarts
Step 3: Review the Resource State History
Use the Resource State History to see when the JVM Resource changed state before. A Resource that goes to Warning more often, or stays there longer, over days or weeks points to heap exhaustion or a memory leak.
Step 4: Capture a JVM Thread Dump
If free heap stays low and no application event explains it, or the CPU threshold fires at the same time, capture a JVM thread dump to see which threads are busy:
jstack <PID> > threaddump.txt
Step 5: Analyze a Heap Dump if Required
For a persistent memory leak, capture a heap dump for analysis with tools such as Eclipse MAT or VisualVM:
jcmd <PID> GC.heap_dump /tmp/heapdump.hprof
Adjusting the Configuration After Correlation
Once you identify the cause:
- Batch job spikes: Increase the Duration on the Heap Size tab in the JMX Configuration to ignore expected spikes
- Traffic-related spikes: Lower the Warning Threshold % (minimum free heap) to accommodate peak usage, or give the JVM more heap with
-Xmx - Memory leaks: Fix the underlying code issue; threshold changes only hide the problem
Important
Never lower the free heap thresholds to silence a state change without understanding the root cause. A lower Error threshold leaves less time to act before a real OutOfMemoryError.
Next Steps
Environment-Specific Thresholds
Polling Frequency
Related Topics
Heap Used vs Committed
Supported Garbage Collectors
Resource State History
JMX Troubleshooting Overview