Understanding JVM Memory architecture and guidelines and tools for troubleshooting

Introduction

Have you ever faced memory issues in your Java applications? Whether it's an OutOfMemoryError or performance degradation, understanding the JVM's memory management is crucial for effective troubleshooting. In this post, we'll explore the basic JVM memory architecture, common issues, and tools to diagnose and resolve memory-related problems.

JVM Memory architecture

Replay

Below are the key components of the JVM memory architecture:

• Heap: stores Java objects. Any object created in Java is allocated in the heap. Managed by the GC.
• Metaspace: Replaces PermGen in Java 8+, stores class metadata including methods, constant pools, symbols etc. This is loaded by class loaders and is managed by the GC.
• Code Cache: Holds JIT-compiled native code. Used for performance optimization.
• Internal JVM Allocations: Includes thread stacks, thread-local storage, loaded jars, native libraries etc.
• PC Register: Holds the address of the current instruction being executed. One per thread.
• Native Memory: Used by native methods (JNI) and NIO allocations.

Memory that gets used by the Java process is the sum of the memory used for all the above components. When I started working with Java, I used to think that the heap is the only memory area and used to wonder why the memory usage of the Java process is always more than the heap size. Heap is just one of the components of the Java process memory.

Garbage Collection (GC) is a process of automatic memory management in Java. It helps reclaim memory by removing objects that are no longer in use. The JVM uses different GC algorithms to manage memory efficiently, such as Serial, Parallel, CMS, G1, and ZGC. Each algorithm has its own strengths and weaknesses, and the choice of GC can significantly impact application performance.

Java Heap Structure

Heap is divided into two main areas:

• Young Generation: Newly allocated objects. Collected frequently. The Young Generation is further divided into:
  • Eden Space: Where new objects are allocated.
  • Survivor Spaces: Two survivor spaces (S0 and S1) where objects are moved from Eden after surviving a minor GC.
• Old Generation: Long-lived objects. Collected less often. Objects from Survivor spaces that survive multiple GC cycles are promoted to the Old Generation.

Replay

Object Lifecycle Through Generational GC

Replay

The diagram above illustrates how objects move through different memory areas in the JVM during their lifecycle:

1. New objects are allocated in Eden space

   • All newly created objects initially go into Eden space
   • Eden space is designed for rapid object allocation and collection

2. When Eden fills up, a minor GC is triggered

   • JVM identifies live objects that are still referenced
   • Unreferenced objects are removed (garbage collected)

3. Live objects from Eden move to one of the survivor spaces (S0)

   • Eden is then cleared and new allocations continue

4. At the next minor GC, live objects from Eden and S0 move to S1; S0 is cleared

   • The survivor spaces alternate as the destination each cycle
   • Objects have a "tenuring threshold" (age counter)

5. Survivor spaces alternate roles in each minor GC cycle

   • One survivor space is always empty after a GC
   • Age counter increases when objects survive a GC

6. Objects that survive multiple GC cycles are promoted to the Old Generation

   • Old Generation is collected less frequently (major GC/full GC)
   • Objects reaching their age threshold are moved to Old Generation

This generational garbage collection approach is based on the empirical observation that most objects die young. By separating short-lived and long-lived objects, the JVM can optimize garbage collection and improve overall application performance.

Thread stack memory and heap interaction

Replay

• Each thread has its own stack that contains:

  • Method calls (frames)
  • Local variables
  • References to objects (memory addresses)

• All objects are created in the shared heap:

  • The heap is common to all threads
  • Objects in the heap can be accessed by any thread that has a reference
  • Thread stacks only store references (addresses) to the actual objects
  • When a local variable goes out of scope, the reference is removed but the object remains in the heap until garbage collected

• Memory management implications:

  • Stack memory is automatically reclaimed when a method returns
  • Heap memory is managed by the Garbage Collector
  • Objects remain in the heap as long as there's at least one reference to them
  • Thread communication often happens through shared objects in the heap

Now that we have a basic understanding of JVM memory sections, let's look at some common issues that can arise in these areas and how to troubleshoot them.

Common reasons for JVM memory issues

• Memory Leaks: Objects are retained longer than necessary, preventing garbage collection. For example even though the object is no longer required for the program, it is still reachable through static references or collection.
• High Object Creation Rate: Excessive object creation can lead to frequent garbage collections and performance degradation.
• Improperly Sized Memory Sections: Too small a heap can lead to frequent GC, while too large a heap can lead to long GC pauses.
• Improper GC Configuration: Incorrect GC settings can lead to inefficient memory management. Although the default G1 GC is usually sufficient, sometimes you may need to tune it or change to a different GC for specific workloads.
• Native Memory Leaks: Memory allocated outside the JVM (e.g., via JNI) is not managed by the GC, leading to leaks if not released properly.
• Excessive Thread Stacks: Each thread consumes memory for its stack. Too many threads can lead to OOM. With Virtual Threads in Java 21, the stack size is much smaller, but still needs to be monitored.
• Large Object Allocation: Allocating large objects can lead to fragmentation and inefficient memory usage.
• Excessive class loading: Loading too many classes can lead to high metaspace usage, especially with dynamic class loading.
• Excessive use of finalizers: Finalizers can delay object reclamation, leading to increased memory usage.

Impact of Memory Issues

• OutOfMemoryError (OOM): The JVM cannot allocate memory, leading to application crashes. You can get OOM errors when any of the following areas run out of memory:
  • Heap
  • Metaspace
  • Native Memory
• Performance Degradation: High GC activity, long pauses, and slow response times.

Troubleshooting Java Heap Memory Issues

Below are some common troubleshooting steps for Java heap memory issues:

• 1. Monitor heap usage over time using tools like JVisualVM, JConsole, or Java Mission Control (JMC).
• 2. Adjust heap size if needed (-Xms, -Xmx). For example, -Xms512m -Xmx2g.
• 3. If heap continues to grow, investigate for memory leaks using heap dumps and analysis tools like Eclipse MAT.

JConsole is a tool included as part of JDK and can be used to monitor heap usage, thread activity, and other JVM metrics. It provides a graphical interface for monitoring and managing Java applications. You can start it by running jconsole in JDK bin directory. After that you can connect to the JVM you want to monitor.

JVisual VM is another tool that provides a graphical interface for monitoring and profiling Java applications. It allows you to visualize memory usage, CPU usage, and thread activity in real-time. You can download it from https://visualvm.github.io/download

Java Mission Control (JMC) is a powerful monitoring and profiling suite which works seamlessly with Java Flight Recorder (JFR) to create a comprehensive toolchain for analyzing Java application performance. JFR collects detailed low-level runtime data from the JVM with minimal overhead, while JMC provides an intuitive interface to analyze this data. Together, they enable developers to diagnose performance issues, memory leaks, and other runtime problems in both development and production environments. You can download JMC from Oracle's website.

Profiling Tools

You can use profiling tools like Visual VM to identify potential memory leaks. Generational count shown in the below profiling screenshot can help us identify the objects not getting garbage collected. Generational count is the number of times an object has survived a garbage collection cycle. If the generational count is high, it indicates that the object is long-lived and may be a candidate for memory leaks.

All these 3 tools connect to JVM via JMX (Java Management Extensions). All this profiling data is available via JMX.

Troubleshooting Metaspace Memory Issues

• 1. Monitor metaspace usage.
• 2. Increase metaspace size if needed (-XX:MaxMetaspaceSize).

JConsole, JVisual VM, and JDK Mission Control can be used to monitor metaspace usage.

Troubleshooting Native Memory Issues

• 1. Monitor native memory usage using tools like jcmd, jmap, or Native Memory Tracking (NMT).
• 2. Use -XX:NativeMemoryTracking=summary|detail to track native memory usage.
• 3. Investigate for native memory leaks (e.g., JNI, NIO).

Collecting Diagnostic Data

Data Type	Tool/Source	Use Case
GC Logs	JVM options (-Xlog:gc*)	Analyze GC frequency, pause times, trends
Heap Dumps	jcmd, jmap, JMC	Analyze object retention, leaks
Heap Histograms	jcmd, jmap	Quick view of object counts/sizes
Java Flight Recorder	JFR, JMC	Profile allocations, memory leaks
Native Memory Tracking	NMT (-XX:NativeMemoryTracking=summary)	Track JVM native allocations
OS Tools	ps,top, pmap, perfmon	Monitor total process/native memory
Core Files	OS, gdb	Deep native memory analysis

GC Log Analysis

Will provide insights into:

• GC frequency and duration
• Heap usage before and after GC
• Pause times
• Use -Xlog:gc* (JDK 9+) or -XX:+PrintGCDetails (JDK 8) to collect logs.
• Look for frequent full GCs, long pauses, or heap not shrinking after GC.

Sample logs below:

[8.257s][info][gc,start       ] GC(26) Pause Young (Mixed) (G1 Evacuation Pause)
[8.257s][info][gc,task        ] GC(26) Using 8 workers of 9 for evacuation
[8.264s][info][gc,phases      ] GC(26)   Pre Evacuate Collection Set: 0.09ms
[8.264s][info][gc,phases      ] GC(26)   Merge Heap Roots: 0.07ms
[8.264s][info][gc,phases      ] GC(26)   Evacuate Collection Set: 6.99ms
[8.264s][info][gc,phases      ] GC(26)   Post Evacuate Collection Set: 0.21ms
[8.264s][info][gc,phases      ] GC(26)   Other: 0.05ms
[8.264s][info][gc,heap        ] GC(26) Eden regions: 16->0(12)
[8.264s][info][gc,heap        ] GC(26) Survivor regions: 4->3(3)
[8.264s][info][gc,heap        ] GC(26) Old regions: 39->49
[8.264s][info][gc,heap        ] GC(26) Humongous regions: 0->0
[8.264s][info][gc,metaspace   ] GC(26) Metaspace: 75330K(75840K)->75330K(75840K) NonClass: 67338K(67648K)->67338K(67648K) Class: 7992K(8192K)->7992K(8192K)
[8.264s][info][gc             ] GC(26) Pause Young (Mixed) (G1 Evacuation Pause) 230M->204M(348M) 7.487ms
[8.264s][info][gc,cpu         ] GC(26) User=0.05s Sys=0.01s Real=0.00s

We can also use Visualization tools (JMC, GCViewer, GCEasy) which will help spot trends and issues quickly.

Heap Dumps & Analysis

• We can get heap dump from a running JVM process using jmap or jcmd.
• Always set -XX:+HeapDumpOnOutOfMemoryError to get a heap dump when OOM occurs.
• Analyze with Eclipse MAT to find top memory consumers and reference paths from GC roots.

jmap -dump:format=b,file=<filename>.hprof <java process id>

For example:
jmap -dump:format=b,file=student-app-heap-dump.hprof 38065

Dumping heap to /Users/codewiz/student-management-app/studentmanagement-api/student-app-heap-dump.hprof ...
Heap dump file created [172972444 bytes in 0.577 secs]

Now we can use Eclipse MAT to analyze the heap dump file.

You can also see the same information in Visual VM.

Java Flight Recorder (JFR)

Java Flight Recorder (JFR) is a tool for collecting diagnostic and profiling data built into the JVM. It collects low-level runtime data with minimal overhead.

• Start with -XX:StartFlightRecording or use jcmd to start recording on a running process.
• Enable the "old object sample" event for memory leak profiling.
• Analyze in JMC(JDK Mission Control) for allocation hotspots and reference paths.

Native Memory Tracking (NMT)

Native Memory Tracking (NMT) is a built-in tool in the JVM that tracks native allocations made by the JVM.

Enable NMT when starting your Java application by adding one of these options:

# For summary-level tracking
java -XX:NativeMemoryTracking=summary -jar your-application.jar

# For detailed tracking (higher overhead)
java -XX:NativeMemoryTracking=detail -jar your-application.jar

Also you can enable NMT in a running JVM using jcmd

Detecting Memory Leaks with NMT

1. Create a baseline early in your application's lifecycle:

   jcmd <pid> VM.native_memory baseline

   This captures the initial memory state for comparison.

2. Monitor changes over time by comparing against the baseline:

   jcmd <pid> VM.native_memory detail.diff

   Run this command periodically to observe memory growth patterns.

3. Analyze the results - Look for consistently growing memory categories across multiple measurements. Native memory leaks often appear as gradual growth that never decreases.

OS Monitoring & Core Files

• Use ps, top, pmap (Linux/macOS), or perfmon, VMMap (Windows) to monitor process memory.
• Core files can be analyzed with gdb to inspect native allocations and leaks.

References

• Java Memory Management
• JVM Troubleshooting Guide

To stay updated with the latest in Java, Spring, and modern software development, follow me for in-depth tutorials and updates:

🔗 Blog 🔗 YouTube 🔗 LinkedIn 🔗 Medium 🔗 Github