10 Java Flight Recorder Techniques That Diagnose Production JVM Problems Without Restarting

I remember the first time I saw a Java application go sideways in production. No debugger, no extra logging, no way to add a line of code without a painful restart. That sinking feeling when you know something is wrong but have no tools to look inside the running JVM. Then I discovered Java Flight Recorder and Mission Control. These two tools are built into the JDK. They let you record everything that happens inside your JVM with almost no overhead. You can start a recording without touching the application. You can stop it later and analyse the data to find the exact root cause of slow responses, memory leaks, or high CPU.

Let me walk you through ten techniques I use regularly when profiling production Java applications. I will keep the language plain. I will show you code snippets you can copy and test yourself.

Starting JFR at Application Boot

The easiest way to get a recording is to add a command-line flag when you start your application. This way the recording begins immediately, even before your main method runs. I use this for every new service I deploy. The flag looks like this:

java -XX:StartFlightRecording=filename=startup.jfr,settings=profile,duration=60s,delay=30s -jar myapp.jar

The recording waits 30 seconds for the application to warm up, then records for 60 seconds. The output file is written to disk when the recording ends. If the application crashes during the recording, the file is still saved. For long-running services, I set up continuous recording that rotates files automatically:

-XX:FlightRecorderOptions:maxchunksize=10m,maxsize=500m,repository=/tmp/jfr

JFR writes data into a ring buffer and flushes to disk in small chunks called “chunk files”. I tell it to keep at most 500 MB of these files in a repository directory. Old files are deleted when the limit is reached. This way I always have the last few hours of profiling data available.

Starting and Stopping Recordings Dynamically

Sometimes you do not want to record from boot. Maybe you only need to capture a problem when it happens. In that case you use jcmd. This is a command-line tool that connects to a running JVM and sends it instructions. You never need to restart the application.

jcmd <pid> JFR.start name=diagnostic duration=120s filename=/tmp/diag.jfr settings=profile

You replace <pid> with the process id of your Java application. This command starts a 2‑minute recording. You can check the status with jcmd <pid> JFR.check. You can stop it early with:

jcmd <pid> JFR.stop name=diagnostic

I have a monitoring script that watches the response time of my endpoints. If the 99th percentile jumps above 500 ms, the script runs the jcmd JFR.start command. I capture exactly the moment of the problem without recording extra data all day long.

Analysing the Flame Graph in JMC

Once you have a recording file, open it in JDK Mission Control. This is a desktop application that comes with the JDK. The first view I always open is the Flame Graph. It lives under the “Code” tab. Each bar represents a method call. The wider the bar, the more CPU time that method consumed. The bars are stacked from bottom to top, so you see the call chain.

Look for methods that are unexpectedly wide. If you see a method from your own code taking 40% of the CPU, that is a strong signal. Right-click on it and select “Show source” if you have debug information attached. I once found a simple log statement inside a hot loop that was formatting a string with String.format for every request. That single line was consuming 15% of the CPU. Removing it solved the latency issue.

Inspecting Memory Allocation Hot Spots

The “Heap” section in JMC shows you how many objects are allocated per second, what types of objects, and where the garbage collector is spending its time. There is an “Allocation Profiling” tab introduced in JMC 8+. It is like a flame graph but for memory. It shows which code paths create the most objects.

I look for methods that allocate a huge number of temporary objects. For example, every call to a method that creates a new StringBuilder inside a loop, or a method that boxes an integer inside a lambda. JFR can also capture the exact stack trace for every allocation event if you use the allocation event setting:

-XX:StartFlightRecording=settings=profile,events=allocation*

This increases overhead, so I use it only for short recordings. But it is powerful. I once reduced GC pauses by 70% after I saw that 80% of allocations came from a single method that was converting BigDecimal to string for logging that nobody read.

Detecting Long‑Lived Threads and Thread Contention

Open the “Threads” view in JMC. You see a timeline with threads and their states (RUNNABLE, BLOCKED, WAITING). If a thread stays in BLOCKED state for more than a few milliseconds, that means it is waiting for a lock held by another thread. Too many BLOCKED threads means your application is contending for shared resources.

The “Lock Profiles” view shows which locks are contested and which methods hold them. I look for locks held by methods that run for a long time. One common mistake is synchronizing an entire method that does database calls or file I/O. Instead, synchronize only the critical section. JFR can even capture the stack trace of the thread that currently holds the lock, making it easy to find the culprit.

In one production issue, I found that a single synchronized block inside an account‑processing service was causing 90% of threads to block. The lock was held by a method that sent an email – a network call. I moved the email sending outside the synchronized block, and the throughput jumped by a factor of ten.

Analysing Garbage Collection Pauses

The “Garbage Collections” section lists every GC event. I sort by duration and look for full GC events. Full GCs are bad because they stop all application threads. They usually happen because the old generation is too full, or because of concurrent mode failures in G1.

JMC shows the heap usage before and after each collection. If the heap after a young collection is still high, young objects are being promoted to the old generation too early. I check the tenuring threshold and the size of the young generation. I add flags like -XX:NewRatio=2 or -XX:MaxTenuringThreshold=15 to keep objects alive in the young generation longer.

I also look at the “GC Pauses” timeline. If you see frequent short pauses, your application may be generating too many objects. Fix the allocation hot spots first.

Using JFR for Exception and Error Logging

JFR can capture every thrown exception and error, including the full stack trace. This is much cheaper than logging each exception to a file. To enable it, add an event setting:

-XX:StartFlightRecording=events=throwable*

In JMC, open the “Exceptions” tab. You see a histogram of exception types. Double‑click one to see the stack traces. I look for exceptions that are thrown many times per second. Often they are swallowed in a catch block with a log message like “ignored”. Those exceptions still cost CPU time to construct and fill the stack trace.

One time I found that a JSON parser was throwing a JsonParseException on every request because of a malformed field. The exception was caught and logged, but the code continued. Removing that unnecessary exception handling improved performance by 20%.

Profiling the Compiler and Code Cache

The JIT compiler is always working in the background. JFR records which methods are compiled, how long compilation took, and how much code cache is used. Open the “Code” section in JMC and look at the “Compilations” view. If you see many methods being compiled repeatedly (tiered compilation), your code cache may be too small.

You can increase the code cache size with -XX:ReservedCodeCacheSize=256m. JFR also shows methods that are too large to be compiled. Those methods run in interpreted mode, which is slower. Split them into smaller methods or add -XX:-Inline to force inlining.

I once fixed a slow authentication routine by splitting a monster method that had over 800 bytecodes. The JIT compiler refused to inline it. After splitting, both methods were compiled and the whole flow ran three times faster.

Setting Up Continuous Monitoring with JFR

For proactive monitoring, I set up a continuous low‑overhead recording that rotates every hour and keeps the last 24 hours. I use the default settings which have about 1% overhead – acceptable for production. The command:

-XX:StartFlightRecording=filename=/var/log/jfr/recording-$(date +%Y%m%d-%H%M).jfr,maxsize=2g,settings=default

I also have a cron job that removes recordings older than a week. When a problem occurs, I open the recording that covers the timeframe. This is much faster than trying to reproduce the issue. I have found countless bugs this way, from memory leaks in dependency libraries to thread pool misconfigurations.

Correlating JFR Events with External Metrics

Each JFR event has a timestamp with nanosecond precision. I export my application logs with the same timestamp format, usually ISO‑8601 with microseconds. Then I can cross-reference slow HTTP requests with GC pauses, lock contention, or high CPU events.

JMC has an “Open Data” view that lets you export event tables as CSV. I import that into my monitoring dashboard. For deeper correlation, I tag each JFR recording with a unique identifier that I also include in my structured log lines. When a customer reports a slow response at a specific time, I open the corresponding JFR recording and search for that timestamp. I can see exactly what the JVM was doing: which threads were running, which locks were held, which GC was happening.

These ten techniques have saved me weeks of debugging. JFR and JMC are not magic – they give you data. You still need to interpret it. But they replace guesswork with evidence. Start by running a default recording continuously. Then learn to look at the flame graph, the allocation profile, and the thread contention view. The more you use them, the faster you will spot the root cause of production issues.