Why 70% of large-scale legacy modernization projects fail — and how progressive transformation offers a safer, smarter path forward.
The Hidden Risk Beneath Your Business
Across industries, CIOs can see that the very foundations of their long-standing success drivers — such as heritage ERP and customised core systems — have become major liabilities. A recent Gartner report highlights that “More than 70% of recently implemented ERP initiatives will fail to fully meet their original business use case goals. As many as 25% of these will fail catastrophically.”
Why such a disappointing success rate? Legacy systems are frequently too complex to untangle, poorly documented, and deeply intertwined with processes that a few current employees fully understand. In most enterprises, those systems are essentially ‘black boxes’— critical, yet opaque; reliable, yet brittle.
Recent history offers painful proof:
German grocery chain Lidl collaborated with SAP to replace its in-house inventory system with a new SAP ERP solution. The project lasted seven years and, ultimately, failed, costing Lidl a total of around $580 million.
Other well-known examples — Hershey, Nike, Levi Strauss — show that even global brands with deep budgets can stumble when attempting “big bang” replacements.
These failures are not isolated; they are all symptoms of the same disease — underestimating the complexity, hidden dependencies, and knowledge erosion while planning successful “modernizing” long-lived systems.
The 6 Most Costly Mistakes CIOs Must Avoid in Legacy Modernization
Six recurring mistakes stand out when drawing from our work modernizing large enterprise estates and related case studies from the likes of Gartner, McKinsey and MIT Sloan:
1. Treating Modernisation as a One-Time Project
Mistake: A single, massive ERP replacement rarely succeeds. Legacy estates are living organisms, embracing decades of business logic and quick fixes layered over time. A big-bang approach assumes they can be replaced in one fell sweep. They can’t.
Better: Adopt a progressive modernization strategy — incremental positive steps that deliver early wins, reduce risk, and build internal confidence and knowledge with each discrete leap forward.
2. Ignoring the “Tribal Knowledge” Problem
Mistake: Legacy systems often depend on a small group of veteran engineers who understand the system’s inner workings. Documentation is incomplete, and knowledge transfer is fragile. As these specialists retire or move on, organisations lose internal and unique critical expertise (AKA Tribal Knowledge).
Better: Use technology, from knowledge-capture tools to AI-driven documentation techniques that will decode and preserve system logic early in the journey. Then make that knowledge accessible to educate the modernisation engineers.
(Example: Rocket Software and Mitrai’s knowledge-mapping methods for MultiValue and COBOL environments.)
3. Misdiagnosing the reasons for glacial progress
Mistake: Diagnosing a seemingly insurmountable technical challenge as a process mapping issue. In one failed multi-million-dollar ERP migration, it became apparent that the project lacked clear specifications from the outset, lacked a well-defined process and mapping, and was further complicated by unclear interface explanations, which were cited as key causes.
Better: Start with a discovery phase that is designed to explicitly map business processes to code before detailed planning. Understanding what the system actually does today, not what the documentation claims it does. And work out how to improve it from those truths.
4. Neglecting Incremental Value Delivery
Mistake: CIOs often struggle to show ROI until very late in a multi-year programme — eroding CEO-level business support, confidence and continued funding.
Better: Apply agile product thinking and from the outset. Set out to deliver measurable improvements (speed, compliance, automation) every few months. A smart modernisation roadmap treats each completed phase as a product release rather than a minor milestone to something huge.
5. Failing to Align Technology and Business Strategy
Mistake: Too many programmes focus on technical migration rather than strategic capability building. A shiny new ERP or platform means little if it doesn’t align with how the organisation competes and operates. Until you can explain “why?” to the business convincingly, you shouldn’t set off technology-first. (Just because you can doesn’t mean you should).
Better: Modernisation should unlock agility — faster time-to-market, deeper analytics, better customer experiences. Every decision should tie back to a pre-agreed business metric.
6. Choosing the Wrong Modernization Path
Mistake: Many failures stem from choosing an extreme: either a total rewrite or a complete lift-and-shift. From full re-platforming to refactoring or encapsulation, the right approach depends on your system’s risk, criticality, and cultural appetite for change.
Better: Use a structured decision framework — like Mitrai’s 5-Stage Legacy Modernization Framework — to prioritise and assess where to stabilise, simplify, and transform progressively.
Why Progressive Modernisation Works
Gartner calls it “the art of decoupled transformation.” Instead of replacing the whole system, CIOs isolate functions, APIs, and data services, so they can solve problems and deliver modernization in manageable slices.
Key advantages to this approach and philosophy:
Reduced risk because each module delivers value independently.
Continuous learning so that insights revealed and captured from early phases shape later ones.
Budget flexibility means that investment scales logically when aligned with incremental successful outcomes.
Cultural change is encouraged as teams necessarily learn to work consistently and iteratively instead of more rigidly and stressfully in massive, high-stakes project releases.
Progressive modernization isn’t a compromise — it’s a strategy that respects complexity.
It acknowledges that legacy systems are not obstacles to be discarded but assets to be evolved.
Lessons from the Field: Turning Legacy into a Catalyst
In Mitra AI’s experience helping global enterprises modernise legacy estates, three lessons consistently hold true:
Measure Twice before you Cut Once — Map code, data flows, and dependencies before making any change.
Small is strategic — Modernise the smallest viable part that delivers visible business value.
Build organisational confidence — Each successful step creates momentum for the next.
When done right, legacy modernization transforms from a cost centre into an innovation catalyst — enabling AI, analytics, and automation that were once impossible on monolithic architectures.
Final Thought
Legacy systems fail not because technology evolves — but because organizations forget how they work. Modernization is not about replacing the past; it’s about understanding it deeply enough to evolve it safely. Avoid the six costly mistakes, embrace a progressive approach, and your next transformation can become a showcase of value realised, not cost written off.
By Mark Lister
Head of Business, UK Mitra Al
