Delivering large-scale, new IT systems is always a challenge. A glance around the world reveals many instances of projects getting into difficulties. Because of greater openness, problems in the public sector get the most publicity. But the private sector is not without its share of troubled projects; it’s just better at hiding them.
Roughly speaking, we’re looking at projects to introduce mission-critical systems of a significant scale, requiring over 75 man-years or so of effort. The systems may be newly-written or some form of package. Some may be new applications but many more are likely to be replacing an existing system.
Disappointment is all too frequent. For example, one study on project size, by Saur et al, published in CACM, Volume 50, Number 11, November 2007, shows that at about 100 man-years, the chance of seriously missing one or more of the targets of time, budget and functionality is about 80%. Above 200 man-years, the probability is 100%. Data published by Standish show similar results.
Unsurprisingly, there is a vast amount of advice on offer about how to improve the chances of success. I want to look at just one specific point: the perils of the Big Bang – doing everything in one go. There are two sets of activity in introducing a new system: developing it and bringing it into production – the cutover. The Big Bang approach covers the development phase and may also encompass cutover. Let’s look at each in turn.
Development may mean designing and writing a new application. If it is to replace an existing large-scale application, the development effort and elapsed time required will be significant unless a massively improved development tool is available. What happens to new business requirements during this time? Are they added to the old system and the new, complicating the development process? Or do they wait for the new, thus depriving the business of what it needs in the – probably extended – interim?
Packages are tempting, as they are assumed to avoid the development problem. They are a good option for applications that do not differentiate the business in any way. But considerable caution is necessary for applications that are critical to deliver the organisation’s core business. In many cases, the package may have to be adapted to fit the organisation’s business process or the processes adapted to the package. Both are likely to be time-consuming and expensive, raising the same concerns about handling new requirements.
When the new system is ready, probably rather later than expected, it has to be cutover. This could be done in phases, by bringing on users in groups for instance. Or the Big Bang could continue, with a switch for all users at once. The latter approach is risky, unless very extensive volume testing has been done. Unexpected problems will have a significant effect on the organisation, possibly resulting in extended downtime and loss of revenue or reputation.
But why are so many big projects undertaken without due recognition of the potential problems? Why is there so much unwarranted optimism, especially when it comes to replacing existing systems? The industry is replete with examples of highly expensive migrations, far exceeding original cost projections and implementation timescale expectations. Even as the delays increase, the projects still continue because those who authorised them in the first place do not wish to admit defeat.
And yet there are alternatives. A far better approach is to develop new functions in stages, and integrate them with the existing system, resulting in an expanded application, based on service-oriented architecture. Selected existing functions can be progressively moved into a new environment, provided there is a business case; the approach is gradual and in smaller steps, thus minimising risk.