​​A Pipeline Approach to Language Migrations

Valuable software often outlives the technologies used to implement it. Business applications that capture the unique rules, processes, and special cases of an organization — such as inventory, accounting, or sales management — can remain relevant for decades. The business problem they solve is still there, but the technology chosen thirty years ago rarely remains the best way to maintain and extend the system. Think about watching an old movie: the story remains valuable, but watching it today on a cathode-ray TV would feel outdated. We want to preserve the value of the business solution while making it accessible through modern technology.

Why language migrations matter: legacy system constraints, skill shortages, and modernization goals 

When systems are implemented in legacy technologies such as RPG, COBOL, Visual Basic 6, or SAS, organizations face several obstacles:

• Developers with the required skills become scarce. The ones available are closer and closer to retiring, and the talent pool is very limited 
• Ecosystems are limited compared to modern languages. Forget getting access to millions of libraries and frameworks from Maven Central, NPM, or NuGet
• Integration with contemporary platforms becomes increasingly difficult
• Development paradigms are obsolete and not designed to develop very large systems (e.g., lack of modularization abstractions, usage of GOTO statements)
• Tooling and environments are also less productive than what is available today for modern languages. The result is slower development and higher costs.

Clearly, staying on these technologies becomes unsustainable.

Common pitfalls of monolithic “big bang” migrations

A common but risky response to the problems of an aging codebase is to discard the legacy system and attempt a full rewrite in a modern language. These “big bang” rewrites have a long history of failure, as they underestimate the complexity and subtle knowledge embedded in the existing code. For example, your system may contain rules to decide when to order parts from a supplier, and this logic considers countless corner cases like minimum quantities, delivery times, and holidays. While it may seem simple to replicate, the devil is in the details, and the true complexity is often underestimated.

The problem is that the vast majority of developers out there would not think of any solution other than rewriting everything from scratch in some modern language.

Well, that was true until ChatGPT came out. Now people can be tempted to just throw LLMs at the problem. Which is not a good idea. LLMs are about creativity, and while creativity is great if you are generating a story or a poem, it is not a great fit when you want to migrate a few millions lines of code and you cannot afford any deviation.

There is another solution which is not so widely known, and that is the usage of algorithmic transpilers: software that takes code written in a language and produces quality code written in another programming language. This is how TypeScript gets transformed into JavaScript, for example. 

Transpilers are similar in nature to compilers: they transform code in predictable and transparent ways. They work reliably: every day, they translate millions and millions of lines of code, without a single problem. By using them, migrations can be designed as systematic engineering processes: decomposed into verifiable steps that preserve functionality, improve maintainability, and produce idiomatic code in the target language.

This article explores how to structure a pipeline for automated language migrations, showing how intermediate models, semantic enrichment, and open standards like LionWeb can support practical, step-by-step transformations. The goal is to demystify the process and highlight how organizations can preserve decades of business value while moving away from obsolete technologies.

The Challenges of Code Migrations

Automatically migrating code from one programming language to another is not just a matter of syntax conversion. It involves a set of deep technical challenges that determine whether the migration will succeed or fail. Three of the most significant are:

1. Verifying equivalence
2. Producing idiomatic code
3. Coping with paradigm differences

Verifying equivalence

The fundamental goal of a migration is to ensure that the new system produces the same results, from a business perspective, as the original one. Visible equivalence is what matters: the end results must match, even if the internal mechanisms differ. For instance, the original system might rely on temporary files or ad-hoc caching, while the migrated version may do things differently internally and that is perfectly okay — what matters is that the final reports and outputs remain identical.

Equivalence verification remains one of the hardest open problems in migrations. In practice, two complementary strategies can be used:

• System observation and replay: record the behavior of the legacy system given a known initial state, then replay the same interactions on the migrated system and compare outputs. While attractive in theory, this requires building test environments, preparing controlled datasets, and orchestrating repeatable scenarios, which can be complex in practice.
• Functional testing: typically legacy systems have no existing tests, so new ones must be written. High-level acceptance tests, typically expressed in a language such as Gherkin, are particularly valuable. They allow analysts and domain experts — not just developers — to validate business-level behavior, while simultaneously documenting the system’s expected outcomes for the future. LLMs may help draft such high-level tests, but they would still need human revision and approval.

The first approach requires less effort, while the second brings more long-term benefits as it surfaces the implicit knowledge trapped in code into a format that stakeholders can inspect, learn from and re-examine over time.

In any case, some degree of manual validation is unavoidable. While it may be unpopular, there is just no way to avoid some effort in verifying the result of a migration. Manual validation is a necessity. The key is to limit the amount of effort we need to put into this activity by complementing it with automated testing based on the two strategies presented.

The final point to consider is that verification is not just about ensuring the new system works correctly but also about building trust and gaining support. If the organization is not confident in the new system, the risk is that it will not be adopted, which would cause the failure of the migration project.

Producing idiomatic code

Poorly executed migrations often create “transliterated” code — syntactically valid in the target language but stylistically foreign. This is how COBOL-to-Java efforts gave rise to “JOBOL“: Java code that looked and behaved like COBOL, complete with unmaintainable structures.

The cause is usually a translation performed at a level of abstraction that is too low, preserving every detail of the original implementation instead of adapting it to target-language conventions. The solution is to raise the level of abstraction: identify code idioms in the source (e.g., specific looping constructs in COBOL) and map them to idiomatic constructs in the target (e.g., streams and for-each loops in Java). This can be integrated into the migration itself or applied as iterative refactoring steps.

The key is to migrate semantics and intent, not syntax. Without this, the migrated system risks being technically correct but practically unmaintainable.

Coping with language paradigm differences (procedural → object-oriented; static → dynamic) 

The hardest challenge arises when migrating between languages with fundamentally different paradigms. Moving from procedural to object-oriented code, or from static to dynamic typing, requires more than structural rewriting.

Some paradigm shifts cannot be handled automatically at all — for example, translating general Python logic directly into SQL, where the expressive power simply does not match. Even more tractable shifts, such as procedural to object-oriented, are only partially automatable. A migration tool may detect recurring patterns — such as groups of procedures that naturally map to methods in a class implementing a common interface — but full refactoring usually requires manual intervention.

The practical strategy here is often hybrid: perform the migration in a way that produces correct, maintainable code in the new language, but accept that deeper paradigm alignment will evolve incrementally through refactoring. Modern IDEs and refactoring tools make this feasible in a way it was not in past decades. After the language migration is completed, the code can continue to become more readable and maintainable over time.

Typical Migrations in Demand

While COBOL-to-Java migrations have historically attracted the most attention, they are not the only scenario where organizations need automated solutions. Several other migrations are increasingly relevant today:

RPG to Python

RPG, once widely used to build ERP systems for medium-sized businesses, is a prime candidate for migration. The language remains entrenched in many IBM i (AS/400) environments, but the talent pool of RPG developers has become vanishingly small. Migrating to Python offers two major advantages: access to a much larger ecosystem of libraries and frameworks, and a language that is more expressive and adaptable for implementing business logic.

4GLs to Java or C#

Many fourth-generation languages (4GLs) like IBM EGL failed to achieve long-term adoption, leaving organizations with critical systems on obsolete platforms. For these companies, migration to a modern ecosystem like Java or C# is unavoidable to ensure maintainability.

Other Common Scenarios

• SQL dialect migrations: Many systems rely on vendor-specific SQL dialects (e.g., Teradata SQL, Oracle PL/SQL) that organizations want to replace with open-source or more portable alternatives such as PostgreSQL.
• Visual Basic 6 to JavaScript or C#: Legacy desktop applications written in VB6 often need to be migrated to web-based solutions, making JavaScript or C# common targets.
• SAS to Python: Organizations that have long depended on SAS for statistical analysis and data processing increasingly face high licensing costs and limited flexibility. Python, with its mature data science ecosystem (pandas, NumPy, scikit-learn, TensorFlow), has become the preferred target for migration.

Scale of Typical Migrations

These migrations usually involve large-scale systems: hundreds of thousands to tens of millions of lines of code. Several factors explain this scale:

• Decades of accumulation: Systems that have been evolving for 20–40 years inevitably grow massive.
• Copy-and-modify development practices: In environments with poor tooling and no automated testing, developers often extend systems by duplicating existing code and making minimal modifications.
• Unused but unpruned code: Legacy systems rarely include reliable mechanisms to detect dead code.

The scale and complexity of these migrations mean that manual rewrites are impractical. And this way automated migrations become the only feasible path forward.

Structuring a Migration Pipeline

Migrations are inherently complex, and any attempt to handle them through a single, monolithic tool quickly leads to excessive complexity. A more sustainable and effective approach is to design the process as a pipeline of successive, verifiable stages, where each stage has a clear responsibility and can be understood in isolation. This follows the classic principle of divide and conquer: break down a large, complex problem into smaller subproblems that can be solved, tested, and validated independently.

Why a Pipeline Approach?

• Isolation and testing: each stage of the pipeline can be validated on its own, making debugging and quality control easier.
• Reusability: common pipeline components (e.g., parsers, analyzers) can be reused across different migrations, improving efficiency and lowering costs. For example, an RPG parser could be shared between an RPG-to-Python and an RPG-to-Java migration pipeline. 
• Tailoring: differently from using a monolithic, off-the-shelf transpiler, the solution can be adapted according to the needs of each organization. Thanks to the fact the components can be reused, this does not cause efforts or costs to skyrocket.
• Parallel development: different teams can work on separate stages in parallel, accelerating progress.
• Progressive delivery: the pipeline allows partial migrations to be tested and validated incrementally, rather than waiting for an “all-or-nothing” result.

Instead of treating the system as a black box and rewriting everything at once, the pipeline provides a controlled flow: original files enter, are analyzed and transformed step by step, and finally emerge as valid, idiomatic files in the target language.

By structuring migrations as pipelines, organizations not only reduce risk but also create a repeatable engineering process. We get a systematic transformation mechanism, where each stage can be inspected, tested, and improved.

Iterative Validation and Feedback Loops

A naïve approach would be to feed all source files into the pipeline and wait for a complete translated system to come out the other end. In practice, this rarely works. Instead, migrations should proceed iteratively, translating subsets of files in carefully chosen order, validating them, and feeding lessons learned back into the pipeline.

This iterative process requires a migration plan that defines the order in which files are processed. Two principles guide the generation of the plan:

1. Dependency order: migrate files that have no dependencies first (e.g., the files defining some data structures), followed by files that rely on them (e.g., programs using those data structures). In real projects, clusters of mutually dependent files often appear, which must be migrated together.
2. Complexity order: start with the simplest files and gradually move toward the most complex. This allows the pipeline to handle basic constructs early, then progressively expand support as more advanced features appear. Because of this, the first results can be produced quickly. Some migrated files can be given to those who can validate them and feedback can start flowing in.

The outcome is an incremental migration path. At each step, partial results can be compiled, executed, and tested. 

The Role of LionWeb

In a migration pipeline, each step — parsing, semantic enrichment, analysis, code generation — produces a new state of the system. These states are referred to as snapshots. It is convenient to express these snapshots using a language-neutral, open format that many tools can produce and consume.

This is where LionWeb comes in. LionWeb is an open-source initiative, created with contributions from well-known language engineering experts from many organizations including JetBrains, Canon, F1re, Itemis, and Strumenta. It is aimed at defining a standard representation for abstract syntax trees (ASTs).

Using LionWeb as the exchange format brings several advantages:

Language-Neutral Representation

Each pipeline stage can emit its output in LionWeb format, which subsequent stages consume. Because LionWeb has implementations across multiple platforms — Java, Python, TypeScript, C# — pipeline components can be written in whichever language is most suitable. This makes it possible, for example, to parse legacy code in Java, analyze it with a Python-based tool, and visualize results in a TypeScript dashboard, because we can use the same interchange format from each of these languages.

Interoperability Between Tools and Authors

By adopting a shared open standard, migration pipelines are not limited to custom-built components. Tools written by different authors, or even developed for different migration projects, can be combined. This interoperability reduces duplication of effort and makes migration pipelines more modular and reusable across contexts.

Inspecting and Debugging Intermediate Snapshots

LionWeb also enables inspection of intermediate snapshots, which is critical for debugging and validation. Snapshots can be:

• Explored through dedicated tools such as the LionWeb Server Admin UI, where an intermediate AST can be uploaded, navigated, and checked interactively.
• Analyzed programmatically in Python notebooks or scripts, where engineers can load snapshots, extract statistics, or visualize specific aspects of the code.
• Used for monitoring progress, for example by creating dashboards that track how much of the codebase has been parsed, analyzed, or successfully migrated.

This ability to pause, inspect, and verify intermediate states is a decisive advantage over opaque “black box” migration tools. With LionWeb, the pipeline becomes transparent: every stage’s output is accessible, testable, and open to independent analysis.

Core Pipeline Stages

The purpose of the pipeline is to start from the raw information contained in the legacy system, refine the understanding step by step, and gradually move toward a model of the target system that is complete enough to generate high-quality code. 

Parsing

The first step is parsing, i.e., producing ASTs for each of the source files.

Legacy systems rarely consist of a single language or file type. For example, an RPG codebase may include:

• RPG source files for business logic,
• DDS files defining data structures,
• CL scripts orchestrating program execution,
• occasionally even COBOL fragments or assembly routines.

Each of these requires its own parser. The parsers produce abstract syntax trees (ASTs), which are exported into LionWeb format. The result is a forest of ASTs, each representing one source file. Each of these AST may be produced by a different parser, but if all the parsers have support for LionWeb, they can all be processed uniformly from successive stages of the pipeline. At this stage the trees are purely syntactic: they capture structure, but not yet relationships. This is what is provided by the next step.

Semantic Enrichment

The next step is semantic enrichment, which combines symbol resolution and type inference. These two activities cannot be separated in practice.

Take a Java example: to resolve a.b.c, we need to know the type of a.b. To determine that type, we must first resolve b inside a. The resolution of c depends on the class determined by the type of a.b, which may involve inheritance and field lookups. Symbol resolution and type inference form a tightly coupled chain of reasoning, so they must be tackled together.

Semantic enrichment should also handle cross-language and cross-file links, which are common in legacy systems. For example:

• a CL script calling an RPG program,
• an RPG program accessing a DDS-defined data structure,
• different RPG modules sharing variables.

Once this step is complete, the previously isolated ASTs are connected by reference edges, turning them into a coherent, cross-referenced model of the system.

Analysis

After enrichment, the models can be analyzed to detect specific patterns and structures. The exact analyses depend on the source language.

• Goto analysis in RPG: GOTOs can represent structured control flow when examined carefully. For example, a GOTO jumping backward to a tag with no other references may correspond to a while-loop. Others may translate to break or continue. We can classify them and tag the corresponding AST node with a specific annotation. In further stages, we will look at such annotations to decide how to translate each piece of code.
• Overlay analysis in RPG data structures: RPG allows fields to overlay the same memory area. This makes direct translation into higher-level languages difficult. By analyzing overlays, we can categorize them (safe/simple vs. complex/unsafe) and tag nodes with annotations for later handling.
• Pattern recognition: recurring idioms can be marked in the AST. For example, in Java, a field + getter + setter can be recognized as being conceptually part of a property. We can therefore add corresponding annotations. 

Each analysis step enriches the ASTs with LionWeb annotations — custom tags that encode an insight about that piece of code. The models become progressively richer, capturing not only syntax and semantics but also higher-level patterns.

Transformation

With enriched and annotated models, the pipeline applies transformation rules to produce ASTs in the target language.

Transformations can be:

• Simple, one-to-one mappings (e.g., an if-statement in RPG is converted to an if-statement in Java).
• Complex, idiom-based rewrites driven by annotations (e.g., a combination of a field, a getter and a setter within a JavaBean can be translated into a Kotlin property).

The annotations produced by the analysis stage help put in place all the information we need when we arrive at this point, so that the transformations can be straightforward. 

Refinement

Optionally, the target ASTs undergo refinement to make the code idiomatic:

• Adding the necessary imports, based on the code contained in the AST,
• Reordering members (fields before methods, private before public),
• Normalizing naming conventions,
• Doing automatic incremental refactoring (e.g., removing generated methods no one is using)

This stage is about polishing and increasing maintainability, ensuring the output looks like code a human would have written.

Code Generation

Finally, the refined ASTs are passed to the code generator, which serializes them into actual source files. At this point we obtain valid Java, Python, or C# files that can be compiled, executed, and tested like any manually written code.

Best Practices and Final Considerations

The pipeline approach provides several practical advantages that make large-scale migrations more reliable and manageable. By structuring the process into well-defined stages, teams gain testability, measurability, and modularity — three properties that are essential for projects that typically last months or even years.

Testing Each Stage in Isolation

One of the main benefits of a pipeline is that each stage can be tested independently. For example:

• A parsing step can be tested against a corpus of files checking for parsing errors and validating the ASTs produced.
• Semantic enrichment can be tested by checking symbol resolution statistics or verifying that references and types are consistently linked.
• Analysis passes, such as GOTO classification, can be validated each one separately, verifying they add the right annotations in specific cases.

This isolation makes debugging easier, distributes work more effectively across teams, and builds confidence in the correctness of the pipeline as it evolves.

Monitoring Progress

Because migrations run over long timeframes, progress tracking is critical. Each stage produces measurable statistics that can be displayed in dashboards:

• Parsing: percentage of files parsed successfully, number of unrecognized constructs.
• Semantic enrichment: percentage of symbols resolved, unresolved reference counts.
• Analyses: proportion of GOTOs classified as loops, breaks, or continues; instances still unclassified.
• Transformations: number of constructs and code idioms supported, number of original files fully transformed.

Tracking these metrics turns migration into a transparent, measurable process rather than a black-box effort. This is quite a change with respect to the typical scenario where one has to rely on the “gut feelings” of the developers involved. While Jim is sure that the project “is progressing at a good pace”, having numbers to support it gives a different level of reassurance.

Building Modular Pipelines

Another advantage is modularity. By standardizing intermediate representations (e.g., LionWeb), different components can be developed independently and combined:

• Parsers based on ANTLR, Tree-sitter, or other frameworks can be adapted to emit LionWeb, and parsers based on Starlasu natively support LionWeb.
• Analysis modules or symbol resolvers can be shared across projects.
• Code generators for different target languages can be swapped or extended.

This creates the potential for a catalogue of reusable migration components, enabling companies to assemble pipelines faster and with higher quality, rather than reinventing every part from scratch.

Future Directions and Open Challenges

While this approach already works well in practice, there are some areas where more research is necessary to improve the state of the art:

• Test generation: in particular, using high-level specifications (e.g., Gherkin) to capture business behavior and make verification more robust.
• Automatic idiom recognition: improving the ability to detect higher-level language idioms in source code, so migrations become increasingly idiom-to-idiom (preserving intent and adapting style) rather than construct-to-construct translations.
• Tooling for refinement and inspection: expanding support for snapshot visualization, debugging, and collaborative validation.

Finally, it is important to emphasize what not to rely on. Over the years, many migration projects have failed due to two approaches:

• Manual rewrites, which burn enormous budgets and often collapse under their own complexity.
• “One-size-fits-all” tools that promise magical push-button migrations but hide opaque processes that cannot be validated.

A new temptation is to rely entirely on LLMs for code migration. While LLMs are powerful for creative tasks, they are not designed for deterministic, large-scale transformations where precision is non-negotiable. Asking an LLM to migrate millions of lines of code and then relying on developers to patch inconsistencies is, in practice, a recipe for failure.

The pipeline approach offers a few valuable qualities: it is systematic, verifiable, and modular. In essence, it turns a migration into an engineering process rather than a gamble.