David Stein, Principal AI Engineer at ServiceTitan, presented Moving Mountains: Migrating Legacy Code in Weeks instead of Years at the very first QCon AI New York 2025.
Stein kicked off his presentation describing how large-scale migrations are synonymous to “moving mountains.” Major architectural changes have historically required a time measured in quarters-of-a-year to years along with an inherent risk in rewriting tremendous amounts of code. While the benefits of a successful migration may be significant, the risks may be equally as significant because migrations may not have always worked out as planned.
He presented an internal case study related to ServiceTitan having migrated their legacy reporting datasets to their new metric platform based on DBT Labs MetricFlow. This included: a reliance on production SQL databases; hundreds of metrics calculated in a monolith application using object-relational mapping; and thousands of lines of C# and complex SQL code. The timeline was quarters-of-a-year to years.
Stein introduced the term, “false summit,” a concept related to hazards that may occur during a migration that may lead to failure. The hard parts of the migration may sometimes stall and those benefits are never realized. Developers are left to ask, “Wait, are we building the right thing?“
The Principle of Acceleration, a concept involving techniques to boost performance by optimizing data flow. As shown below, the old way involved a decision point, early in the process, after establishing the initial proof-of-concepts before investing in the migration. The new way, however, adds a validation step with a standardized set of tools used before the migration begins.
The Assembly Line Pattern, a sequential, automated process for building, testing and deploying applications, often used in CI/CD, is composed of three steps in transforming an application migration:
- Decompose: break the “mountain” into “pebbles.”
- Standardize: create a “physics engine” for automated and instant validation.
- Automate: use of agents to execute tasks.
Stein listed the breakdown of tasks used by ServiceTitan for their heritage migrations, namely: objective; plan; code; test; and release that took them years to complete. Using the Assembly Line Pattern, they can now execute a successful migration in a number of weeks.
Stein discussed a number of challenges with their corresponding solutions along the way:
- The agent incorrectly believes it has succeeded → fix the validation logic.
- The agent gets stuck and can’t make progress → add context and test data; subdivide tasks.
- Small portions of tasks can’t be completed → engineers manually complete the most difficult tasks.
- A better architecture is identified partway → rewrite the migration, goals and validation; start over.
The paradigm shift in lowering the cost of a change in architecture from years to weeks unlocks a new level of agility. Stein mentioned that this has enabled developers to ask themselves new questions for which he offered suggestions that required only weeks to implement:
- “Should we try platform X?” → Just do it and find out.
- “Is this new architecture actually better?” → Build both and compare.
- “Did we make the right call two years ago?” → Remigrate and check.
Stein concluded his presentation by stating that migrations aren’t “mountains” anymore.
