A Hybrid Approach to Painless Java Upgrades using LLMs | HackerNoon

Upgrading legacy Java applications is the dental work of software engineering. You know you have to do it, security audits demand it, performance metrics scream for it, but the thought of hunting down every sun.misc.* import or deprecated API across 500,000 lines of code is paralyzing.

With the rise of GenAI, the immediate instinct is to dump the codebase into an LLM and ask: “What breaks if I upgrade from JDK 11 to JDK 17?”

We tried that. It failed.

In a recent experiment upgrading enterprise systems, pure GenAI approaches resulted in 56% False Positives and missed nearly 80% of actual breaking changes. LLMs are great at predicting tokens, but they are terrible at compiling code in their “heads.”

However, we found a workflow that does work. By pairing Static Analysis (PMD) for detection with GPT-4 for remediation, we reduced the manual effort of incompatibility investigations by 90%.

Here is the engineering guide to building a Hybrid AI Migration Pipeline.

The Problem: Why LLMs Fail at Dependency Analysis

When you upgrade Java (e.g., JDK 8 to 17), breaking changes aren’t just syntax errors. They are structural.

• Deleted Classes: sun.misc.BASE64Encoder is gone.
• Behavior Changes: CharsetEncoder constructors behave differently in JDK 12+.

We initially tried feeding Release Notes and Source Code into GPT-4. The results were messy.

The Pure GenAI Experiment Data

| Metric | Result | Why? |
|—-|—-|—-|
| False Positives | ~56% | The AI flagged methods with similar names but different classes. |
| False Negatives | ~80% | The AI missed issues where packages were deleted but class names remained generic. |

The Verdict: LLMs lack a deep understanding of the Abstract Syntax Tree (AST). They treat code as text, not as a compiled structure. They cannot reliably determine if Encoder.encode() refers to the deprecated library or a custom internal class.

The Solution: The Hybrid Pipeline

To fix this, we need to stop using LLMs for search and start using them for synthesis.

We developed a process where:

1. Static Analysis (PMD) acts as the “Eyes.” It uses strict rules to find exact lines of code with 100% precision.
2. GenAI (GPT-4/Gemini) acts as the “Brain.” It takes the specific context found by PMD and explains how to refactor it.

The Architecture

Step 1: The “Eyes” (Custom PMD Rules)

Instead of asking ChatGPT to “find errors,” we ask it to help us write PMD rules based on the JDK Release Notes. PMD is a source code analyzer that parses Java into an AST.

**The Breaking Change: In JDK 9, sun.misc.BASE64Encoder was removed.

**The Strategy: We write a custom XPath rule in PMD to find this specific import or instantiation.

The PMD Rule (XPath):

import os
import javalang

def analyze_legacy_code(root_dir):
    print(f"🔎 Scanning {root_dir} for JDK 8 -> 17 incompatibilities...n")

    for root, dirs, files in os.walk(root_dir):
        for file in files:
            if file.endswith(".java"):
                file_path = os.path.join(root, file)
                check_file_for_violations(file_path)

def check_file_for_violations(file_path):
    with open(file_path, 'r', encoding='utf-8', errors="ignore") as f:
        content = f.read()

    try:
        # Parse the Java file into an AST (Abstract Syntax Tree)
        tree = javalang.parse.parse(content)

        # 1. Equivalent to XML: //ImportDeclaration[@PackageName="sun.misc"]
        for path, node in tree.filter(javalang.tree.ImportDeclaration):
            if "sun.misc" in node.path:
                print(f"COMBAT ALERT [Import]: {file_path}")
                print(f"   └── Found import: {node.path}")

        # 2. Equivalent to XML: //ClassOrInterfaceType[@Image="BASE64Encoder"]
        # In javalang, 'ReferenceType' or 'ClassCreator' handles type usage
        for path, node in tree.filter(javalang.tree.ReferenceType):
            if node.name == "BASE64Encoder":
                print(f"COMBAT ALERT [Usage]:  {file_path}")
                print(f"   └── Found usage of class: {node.name}")

    except javalang.parser.JavaSyntaxError:
        # Gracefully handle files that might have syntax errors
        print(f"⚠️  Could not parse: {file_path}")

if __name__ == "__main__":
    # Replace with your actual source code path
    SOURCE_DIRECTORY = "./src/main/java"
    analyze_legacy_code(SOURCE_DIRECTORY)

By running this rule, we achieved 0% False Positives and 0% False Negatives. We located every instance instantly.

Step 2: The “Brain” (GenAI Remediation)

Now that we have the exact line number (e.g., UserService.java: Line 42), we can leverage the LLM for what it does best: Coding assistance.

We feed the snippet and the error context to the LLM.

The Prompt:

You are a Senior Java Engineer. 
I am upgrading from JDK 8 to JDK 17. 
Static analysis flagged the following code as incompatible because 'sun.misc.BASE64Encoder' has been removed.

Code Snippet:
String encoded = new sun.misc.BASE64Encoder().encode(str.getBytes());

Task:
1. Explain why this is an error.
2. Provide the modern Java 17 replacement code.

The AI Output (Gemini 2.0 Flash / GPT-4o):

Reason: The sun.misc package was internal and has been encapsulated/removed.

Fix: Use the standard java.util.Base64 introduced in Java 8.

Refactored Code:

import java.util.Base64;
String encoded = Base64.getEncoder().encodeToString(str.getBytes());

Results: GPT-4o vs. Gemini 2.0

We benchmarked different models on their ability to explain and fix these specific static analysis findings.

• Setup: 49 breaking changes identified from Release Notes.
• Task: Explain the error and generate a fix.

| Model | Valid Fixes | Minor Errors | Hallucinations |
|—-|—-|—-|—-|
| GPT-4o | 85% | 12% | 3% |
| Gemini 2.0 Flash | 61% | 39% | 0% |

Insight: While GPT-4o provided more perfect copy-paste solutions, Gemini was surprisingly robust at not hallucinating, though its explanations sometimes lacked depth. Both models, however, are sufficient to guide a junior developer through the fix.

Implementation Guide: How to do this yourself

If you are facing a massive migration, don’t just chat with a bot. Build a pipeline.

1. The MVP Approach

• Target: Select a single module (approx. 500k steps).
• Tooling: Install PMD (Open Source).
• Process:

1. Parse the JDK Release Notes for your target version.
2. Ask an LLM to convert those textual notes into PMD XPath rules.
3. Run PMD against your codebase.
4. Feed the violations into an LLM API to generate a “Migration Report.”

2. Cost Analysis

In our validation, manually investigating 40 potential incompatibilities took a senior developer 2 full days (finding, verifying, researching fixes).

Using the PMD + GenAI workflow:

• Detection: < 1 minute.
• Fix Generation: ~5 minutes (API latency).
• Human Review: 2 hours.
• Total Effort Reduction: ~90%.

Conclusion

GenAI (LLMs) is not a replacement for deterministic tools; it is an accelerator for them.

When dealing with strict compiler rules and legacy code, structure beats probability. Use Static Analysis to find the needle in the haystack, and use GenAI to thread the needle.