ISSN 1566-6379

First published
in 2003

   


   

Paper 1 - Issue 1

Home Papers in this Issue Previous Issues Site Map

    .

Home
About the Journal
Scope
Editorial Board
Submission Guidelines
Call for Papers

 

For information on the European Conference on IT Evaluation, click here

Downloadable documents on this site require Adobe Acrobat Reader (free download here)

The Elusive Nature of Delivering Benefits from IT Investment
Dr Dan Remenyi,  Associate Faculty Henley Management College

1.     Introduction

The amount of corporate funds spent on IT investment has always been controversial. For many years it has been thought by some executives that too much has been spent and not enough return has been obtained from this IT expenditure. Robert Solow (1987) the famous Nobel Prize winning economist has suggested that in business one could see computers everywhere but in the productivity statistics. Brynjolfsson (1993) pointed that there was a productivity paradox which meant that computers where not delivering the value promised. The Economist Newspaper said in 1991, that  the return on IT was so poor that organisations would have done better to have invested that same capital in almost any other part of their businesses.

But was the performance of the investment made in IT really so terribly bad?

The business community wanted to be able to see the benefits of IT in the same way as you see a big colourful brass band coming down the street. They wanted their IT benefits right up in their faces, as the modern idiom would say. However it is not clear why the business community should have expected IT benefits to be so glaringly obvious. The demand for clear and definite benefits may be attributed to management apparently desperate need for control. Keidel (1995) clearly identified this need when he wrote: ‘The overwhelming tendency for management is to be obsessed with control.’

Maybe this expectation was just unreasonable and unrealistic. There are many aspects of business investment where the benefits are really rather subtle, but are no less real or material for that. Examples are corporate head offices, prestige motor cars for executives, and special celebratory conferences to mention only three. Maybe, such is the case with IT investment – perhaps not all of the time but some of the time.

In addition to this, the business community generally demanded that IT benefits be always expressed in terms of financial values i.e. how much money was saved or how much extra money was made as a result of the IT investment.  Some IT investment benefits, however cannot be satisfactorily stated in monetary terms[1]. Nonetheless they are real business benefits and need to be taken into account in any development of the investment evaluation equation.

2.     Economics of information

The measuring and managing of IT benefits is a difficult business challenge which has plagued the IT industry, IT professionals, consultants and academics, for many years. The main reason for this is that despite the considerable amount of research conducted by academics and consultants so far no comprehensive or rigorous economics of information has been developed.

By economics of information[2] is meant a systematic series of concepts and theories that explain the role which information and information systems play in assisting individuals or organisations in their conception, production and delivery of goods and services both in the private and public services. There are probably several reasons why the economics of information has not been developed. One of the most important reasons is that the subject of the economics of information is a very difficult one both from a theoretical and a practical point of view and most practitioners respond to the challenges it offers by either attempting to ignore it, i.e. just get on with the day to day highly pressurised job, or alternatively coping with the problem by understating its importance.  The view which practitioners often tried to assert was that relatively speaking IT expenditure was low and should be seen as an 'act of faith' (Lincoln 1990)[3]. If this view was ever a reasonable one, it is certainly no longer the case as IT investment has become an increasingly important part of corporate capital expenditure.

Top management have begun to insist that much more attention be paid to the economic aspects of information systems than ever before and this has lead to an increasing demand for a comprehensive and reliable IT performance evaluation[4].

3.       Some progress in understanding information systems

During the past decade some progress has been made in understanding and articulating the role and function of information systems in organisations (Keen 1985; Hirschheim and Smithson 1988; Currie 1989; Sherwood-Smith 1989; Scriven 1991:a&b; Symons 1991; House 1993; Walsham 1993; Willcocks and Lester 1993; Farbey et al. 1995; Remenyi et al, 1991, 1994, 1997, 1999; Strassmann 1985,1990, 1997; Ward et al 1997, Irani 1999).

Business computing is nearly 50 years old and for most of that period this subject has been perceived as essentially a technical one[5].  And although technical issues have frequently been of paramount or central importance to the success of business computing, this emphasis has been largely at the expense of a business and management understanding of information systems and what their potential and actual benefits are likely to be.

4.       The problems with IT benefit measurement and management

The past few years has seen a considerable amount of progress in understanding the problems underpinning the difficulties with IT investment benefits and although no clear and simple answer has yet been expounded quite a lot of progress has been made, especially in articulating the problems and producing conceptual frameworks for benefit delivery (Remenyi et al 1997).

There have been four major areas which have contributed to the problems with IT benefit measurement and management. These are:-

1.        Benefits and identifiable performance improvements

2.        The issue of information systems reach

3.        Tangible and intangible benefits

4.        Benefit evolution

4.1     Benefits and identifiable performance improvements

For information systems project success potential benefits need to be identified as early as possible in the systems development cycle. In fact, in an ideal world benefits would be identified and quantified before the information systems development project began. However, although this does happen for some projects it is seldom possible to produce a definitive statement of all the benefits that an information systems development project will produce. In fact a high degree of success at early benefit identification is often quite elusive.

This situation may be complicated when attempts are made to use special IT benefit metrics. No special metrics are required. It is in fact a red herring to measure IT investment success in terms of the utilisation of the system or the number of users or the size of the databases or the number of times they are accessed. This is especially obviously where measures of Web success are concerned. The number of so called “hits” is often offered as evidence of success. However there is not a wide understanding of how a “hit” is defined and even if there were the question of so-what needs to be asked. General business performance metrics such as Net Profit, Return on Investment or Net Present Value are adequate or actually preferable for the identification and measurement of IT benefits.

4.2     Information systems reach

Information systems even when they themselves are simple stand alone systems often, if not usually, play an important integrating type role in organisations. In the words of Evans and Wurster (1997), "information is the glue that holds together the structure of all businesses".

When thought of this way information systems may be seen as inter alia the conduit of that glue or the tracks over which the glue is laid and business activities or processes flow. This integrating type role brings together a number of different corporate issues, problems and resources. Even for the most straight forward information systems applications it is never simple to understand exactly what the results will be of bringing together information about different business issues. There will nearly always be knock-on effects associated with the introduction of any substantial information system especially when such a system has the effect of integrating business processes or even simply integrating reports about business processes.

For example, a payroll system where the primary objective is to automate a series of routine and simple clerical type tasks will frequently be used as an interface to a human resource management application which could include details of training, salary benchmarks, succession planning etc. Also a payroll system may have connections with various costing systems which show actual and standard costs, drawing on the actually amounts paid each week or month. The payroll system may also be interfaced with staff loans accounting. Of course a payroll system will need to be able to transfer data to the corporate general ledgers.  So a relatively simple system such as a payroll can have substantial tentacles which penetrate into a number of different aspects of the organisation. Clearly it is a challenge to visualise all the different identifiable performance improvements such a system can have.

When more complex business applications are considered such as sales order processing, production planning and control or vehicle scheduling it becomes even more difficult to identify all the possible benefits of such systems[6]. The point is that information is at the heart and soul of the business and our ability to build useful information systems directly effect the way the business itself is or may be operated.

The importance of information is well described by Evans and Wurster (1997) when they said:

When managers talk about the value of customer relationships, for example, what they really mean is the proprietary information which they have about their customers and what their customers have about the company and its products. Brands, after all, are nothing but the information - real or imaginary, intellectual or emotional - that consumers have in their heads about a product. And the tools used to build brands - advertising, promotion, and even shelf space - are themselves information or ways of delivering information.

When seen in this light the issue of being able to identify all the benefits in advance becomes a virtually insurmountable challenge. Often the situation is just too complex.

4.3     Tangible and intangible benefits[7]

Some aspects of an information system may produce hard or tangible benefits which will directly improve the performance of the firm, such as reducing costs and will therefore be seen in the accounting number of the organisation as an improvement in profit and perhaps in return on investment (ROI). These benefits are of course relatively easy to identify and to quantify both in physical terms, i.e. the number of people employed or the number of widgets used, and in financial terms, the number of pounds or dollars saved or earned. But other aspects of this system will only create soft or intangible benefits which will improve the general circumstances of the staff and thus make life easier in the organisation but will not directly lead to identifiable performance improvements and as such will not be easily seen in the accounting numbers of the firm.

However, although difficult to be precise about their actual value, especially in financial terms, intangible benefits can make a critical contribution to the success of an organisation. Intangible benefits may often be quantified by measuring instruments such as questionnaires, but it is quite difficult to make a creditable connection between what can be measured with such devices and the impact on the corporate financial results. This whole area of intangible benefits is one of the major problems that makes benefit measurement and management difficult or elusive.

4.4     Benefit evolution

But there is a fourth issue which makes benefit identification and especially early benefit identification even more elusive and that is the propensity for benefits to evolve over time[8]. The benefits of IT are just not stable. Some benefits dry up while other which may originally not have been foreseen, materialise. In short when planning an IT investment it is extremely difficult to look into the future to create a comprehensive catalogue of potentials benefits. No matter how thoroughly the feasibility study or the business case is produced it is usually nearly impossible to foresee all the future ramifications of the proposed process improvement and the accompanying information technology investment in advance. Forecasting is simply notoriously difficult and thus it is perhaps unrealistic to expect a high degree of success at future benefit identification to be anything other than elusive.  This is especially true in the current environment in which business is rapidly changing.

Every information system will have some easy to identify or obvious benefits which will be sustainable over a period of time. However, as the development project proceeds and the ramifications of the system are more fully understood, new ideas about potential benefits will become apparent. This will have been due to the process of creative dialogue between the principal stakeholders, which will bring to light new business processes and practices. In short, potential benefits should not be seen as being static, but rather actually evolve as a greater understanding is gained of the organisation and the role which the system will play in it. Of course it is necessary to point out that in the same way as new benefits surface during the development project, other benefit suggestions which were originally identified may turn out to be illusionary and not really exist.

These three issues are at the centre of why IT benefits have been so difficult to identify and are the prime reasons why benefits and value have been so elusive in the past.

Of course, this problem of business and management understanding and thus of computer evaluation has not been purely limited to individual systems. A lack of understanding has also applied to the more general area of the whole information systems function itself. According to Lacity and Hirschheim (1995):

The problem is that meaningful measures of departmental efficiency do not exist for IS.

Fortunately we are seeing some changes to this unsatisfactory state of affairs. For the past decade business executives, as opposed to information systems executives, have been demanding, by expressing their dissatisfaction at the way in which information systems departments or functions have been operated, a new approach to the management of information systems. The message is beginning to arrive that information systems is an important aspect of many organisations and new ideas as to how to manage them are well under development and a number of different approaches to this problem are discussed here.

5.     Investment, value and economics

It is now pretty clear that the traditional approach to establishing IT investment benefits amount to looking for them in the wrong place. Thus one of the first issues which needs to be addressed in coming to terms with a new approach to IT management is the fact that IT investment has no direct value in its own right. IT investment has a potential for derived value. Furthermore it is widely agreed that IT benefits are not directly a technology issue as such, but are to do with business initiatives. Therefore, they need to be measured and managed by P&L[9] people focusing on business processes and practices. The value of the IT investment depends entirely upon the way in which it is able to make the organisation more efficient and effective[10].

To understand how this actually works in an organisation it is useful to rethink the role of information systems investment by going back to some fundamental concepts. To use classical economic language an information system is a capital or producer or investment good. A capital or producer or investment good is something that is not acquired or valued for the utility it delivers by itself in its own right. Simply, in economic terms capital goods do not have any intrinsic utility or value in their own right, as a television set, a jacket, a meal in a fine restaurant, listening to a guitar concerto, a tennis racket, a holiday in the sun etc do.

A capital or producer or investment good is desired because it can be used to produce other goods and services, which in turn may offer us utility and value such as the television set, a jacket, a meal or a holiday. Capital or producer or investment goods are essentially tools. A bulldozer is a clear example of a capital or producer or investment good. A bulldozer has no intrinsic value on it's own. In fact to many individuals and organisations a bulldozer could be seen as a huge liability as it takes up much space, is costly to move about and requires expensive maintenance and requires a highly skilled and costly operator. It has to be clearly understood that a bulldozer’s value is only derived as a result of its use, i.e. the hole in the ground, the levelling of the old building or the preparing of the ground for a new road or motorway surface. The value potential in economic terms of the bulldozer is thus linked to the result that may be obtained by its appropriate use[11]. The same principle applies to IT or to information systems and is illustrated in Figure 1 below which shows a similar logic to that developed by Soh and Markus (1995).

 

Figure 1: The relationship between business process and IT investment

5.1     IT and derived value

As a producer good, IT has a derived or second order value which is realised when it is used as a component of an organisational or business process or practice. In fact for the value of IT to be generated or realised, it is necessary that the business process or practice to which it contributes actually improve the effectiveness or efficiency of the enterprise. In as far as these organisational or business processes or practices produce improvements to the business they will, at least in the medium term, positively effect one or more of the corporate performance variables. In turn this will show in the corporate performance indicators and thus will be regarded as delivering value. 

Quite specifically if the process innovation and improvement which is to be supported by an IT investment is to improve productivity then the IT investment will be judged primarily on whether that productivity improvement has been achieved.  An improvement in productivity would mean that more goods and services are produced for the same amount of input or that the same number of products or services are produced for less input. This is essentially a cost or pricing issue (Handy 1998). However such a system might also improve the quality of the products being produced and perhaps also have a positive impact on the morale of the manufacturing workers. If this was the case then it would be appropriate to evaluate the process innovation and improvement on all three of these variables.

On the other hand, if the process innovation and improvement was to improve customer satisfaction then the IT investment needs be judged on whether that has been achieved. An improvement in customer satisfaction could be measured by some instrument such as a ServQual scale. In such a case it might not be all that useful to try to reduce these benefits to monetary values. But nonetheless, these benefits are clearly measurable and they can give the organisation a major competitive advantage in the market place. However such a system might also improve the cost effectiveness of serving customers and if this was the case then it would be appropriate to evaluate this process innovation and improvement also in terms of its financial impact on the organisation.

In practice it will seldom be appropriate to evaluate a process innovation or improvement in terms of one metric alone. Thus this type of evaluation really requires multi-metric analysis and this analysis, needs to be managed by the principal stakeholders.

To ensure that the value of the process innovation or improvement is fully exposed a process approach needs to be taken to its assessment. This is similar to the process perspective suggested by researchers such as Weill (1990), Brynjolfsson and Hitt (1995), Barua et al. (1995), McKeen et al.,(1999) ,and Mooney et al. (1995).  Without this process perspective the realisation and the concurrent research for the benefits is far more difficult.

At the outset of the business intervention the process owners need to be quite specific about the their objectives and goals and these need to be stated in a business case or a value proposition or business model. They need to be quantified where possible and a timetable needs to be established as to when they may realistically be expected to be achieved. The next step is to identify in some detail the exact changes to the current procedures and practices which need to take place, and which individuals will be effected by these changes. Then responsibility has to be allocated for these changes taking place together with the necessary resources including time for training etc. Finally a review mechanism needs to be established, along with a procedure to follow if the proposed changes are not actually happening.

As a consequence of the above it is clear that IT investment only derives its value to the organisation though its business applications. This in turn can only be effected in the hands of the information systems principal stakeholders and it is these individuals who need to manage the investment as well as its evaluation. More than any other factor the success or failure of the IT investment is a function of the skill and commitment of the information systems principal stakeholders.

6.     IT investment as an asset

The question of whether an organisation's management of its IT resources is improved if it regards the funds which are spent on this activity to be in the accumulation of an asset as opposed to simple being part of recurrent expenditure, is an interesting one. It has been argued that by emphasising the asset nature of IT investment the organisation will some how ensure that it is more attentively managed and thus more easily produce more benefits. This argument suggests that by labelling the IT investment as an asset more benefits will in fact be derived.  There does not appear to be much ground for this assertion.  In fact, from the above it may be argued that an information system has no intrinsic value in its own right. This is certainly what Strassman (1990) was saying when he pointed out

a computer is worth only what it can fetch at an auction

This type of argument espoused by Strassman suggests that by itself an information system is nothing more than a sunk cost that has been spent on a collection of hardware, software and communications equipment.  And the cost of this ‘kit’ is normally a very large amount indeed. In fact it may be argued that there is really no overriding reason to see information systems as an asset[12], except for the fact that they are reusable i.e. not instantly consumed. Today many personal computers and other small systems are being written off immediately and thus do not appear in the balance sheet but are treated from an accounting point of view merely as an operating cost.

But once the computer is successfully integrated into a business process then the whole picture changes and to be fair to Strassman he did recognise this in his book. Thus, it is generally agreed that an information system really only acquires value i.e. becomes an asset in the colloquial use of the word, when it is employed in collaboration with other resources as part of a business process or practice that will result in the enhancement of the effectiveness or the efficiency of the organisation. This conclusion begs the question of which business processes or practice should be supported by IT and how should this actually happen.

7.     Processes, practices and people

Business processes or practices are made to function by people working in groups or individuals mostly in line or profit and loss (P&L) positions[13]. In the private sector these people make, sell and support the products or services for which the organisation was created. But this argument does not only apply to profit orientated business. In the public sector governmental or not-for-profit organisations focus on services at the national, regional or local level. It is these line or P&L people who know what is required by their organisations to succeed. These individuals also know how information systems can best support their private or public sector efforts.

In effect these groups or individuals use the information systems as tools or producer goods to achieve organisational results. It is the efforts of these people which makes the IT investment a success or a failure. Furthermore they intrinsically know what benefits are actually being delivered and if the information system should actually be regarded a success.

8.     Looking for benefits and value

The main implication of this derived value of IT notion is that the actual benefits of an IT investment cannot be perceived directly or on their own. Only when IT is coupled with other resources, and especially the principal stakeholders, can any benefits or value be perceived. There is no standard way of combining IT with other resources or with people. Davenport (1997) pointed out the importance of people in information systems success when he wrote:

Information and knowledge are quintessentially human creations, and we will never be good at managing them unless we give people a primary role.

Combinations of information systems, people and other resources are entirely dependent on the context of the business process or practice but in general unless IT is an integral part of a greater program of process innovation and improvement is it quite unlikely that much value or benefits will be derived.

9.     Primary stakeholders

These P&L people are the primary or principal stakeholders of the IT investment. By primary stakeholder of the IT investment is meant the individual or group of people who have the most to gain or the most to lose if the investment is or is  not a success. The characteristic of the principal or primary stakeholder which is of most interest is the fact that he or she or they can directly influence the success or failure of the information systems. Most often the principal or primary stakeholders are in fact the user-owners and it is the user-owners who makes all the difference between success and failure. This view is supported by Strassman (1997) when he said:-

The lack of correlation of information technology spending with financial results has led me to conclude that it is not computers that make the difference, but what people do with them. Elevating computerisation to the level of a magic bullet of this civilisation is a mistake that will find correction in due course. It leads to the diminishing of what matters the most in any enterprise: educated, committed, and imaginative individuals working for organizations that place greater emphasis on people than on technologies.

The recognition of these line people as the principal or primary stakeholders[14] in any information system is a fundamental change in approach or paradigm shift for many organisations. In the ‘old days’ it was naturally thought that the systems belonged to the information systems people.  Having information systems people as systems owners produced the most unsatisfactory state of antagonism between  P&L people and IT people, and has been described as the culture gap (Grindley 1991; Townsend 1984; Dekle 1986;   Butler 1985; Palmer, 1990).

This new approach described here largely eliminates that culture gap. It is however important to point out that this does not in any way diminish the contribution of the information systems professional to the successful use of business computing. It simply changes the locus of responsibility for the IT investment decision and the locus of responsibility for ensuring that the new business process is a success.

10.     The locus of responsibility[15]

Placing the principal or primary stakeholders at the centre of the information systems investment does indeed reposition the locus of responsibility for the success of the information system and put it squarely where it should be with the line mangers and user-owners. There are several reasons why the user-owners need to be centre stage in any information systems investment but by far the most important is that by so doing the chances of the information systems delivering the type of support which is required by the business process or practice, i.e. being relevant are substantially increased. The problems which arise when the user-owners are not regarded as the primary stakeholders are well articulated by Davenport (1997) who said when talking as an information systems professional:

We have spent a great deal of time and money bringing water to the horse, but we don’t even know if he is thirsty, and we have no idea how to get him to drink.

10.1     The user-owner  - not the only stakeholder

But the user-owners are not the only stakeholders in an information systems investment. The group of IT professionals who will work with the technical development of the information systems are clearly stakeholders of some considerable importance. They supply the IT expertise which will make the technology aspects of the new processes work. Traditionally in-house IT professionals have accepted a large part of the responsibility for the success of the IT investment. It was not uncommon for IT professionals to develop information systems for their so-called end-users. Sometimes this was done without adequate consultation, as IT professionals have been known to believe that they knew what the user requirements were, at least as well as the users themselves.  Clearly this was not very satisfactory, and as mentioned above, there have been problems of communications between IT professionals and users, as well as the development of what is sometimes referred to as the culture gap.  Increasingly it is thought that the users-owners need to play a greater role in ensuring the success of the IT investment.

However the IT professional still has a totally indispensable role to play in any IT investment project. The IT professional should be seen as a critical adviser to the users/owners who will ensure that the appropriate technology is acquired and employed. The role of the IT professional has also been complicated by the fact that some organisations have outsourced the supply of IT expertise.

Outsourcing has been a familiar aspect of the supply of IT capabilities and competencies in many organisations over the past years. Few organisations have never used contractors or from time to time employed consultants. However it has only been in recent years that organisations have outsourced all, or the major part of the IT operation. Today organisations increasingly use a portfolio of IT expertise, which includes both internal and external people and organisations. All these sources of IT expertise may be stakeholders in their own right and are shown in Figure 2.

 

Figure 2: The more usual sources of IT expertise

In fact there are usually several of these groups involved as stakeholders in any given IT investment.

The third group of principal or primary stakeholders is the financial managers and administrators. Financial managers and administrators are always stakeholders in any corporate investment as they are instrumental in making the funds available for the purchase of the equipment, etc. Financial managers and administrators will arrange the contracts and ensure that goods are received and that payments are made, etc.

Financial managers and administrators are often involved with the detail of the business case accounting, as users-owners may not be familiar with the costing approaches required. IT investments which are made to improve business processes and practices will often affect the internal controls within the organisation, and for that reason both internal and external auditors may be required to advise on the propriety of the new proposals. In addition, investments often have to be audited and this will require further involvement from auditors.

Figure 3 lists some of the stakeholders that fall into this category.

Figure 3: Some of the financial managers and administrator stakeholders

Understanding the importance of having the full commitment of these three sets of primary stakeholders and having them work closely together is an important key to IT investment benefit realisation (Remenyi et al 1997). In fact the project management teams needs to be able to draw on the expertise of all three sets of stakeholders if the project is to be successfully implemented. Figure 4 shows how the project management team needs to be assembled.

Figure4: The Project team needs to draw on expertise from all three of the stakeholder sets.

10.2     Re-focus the traditional role of the IT professional

Thus the traditional role of the information systems professional needs to be re-focused. They need to become primarily advisors and educators. However the information systems professionals, of course, do not lose all their action-orientated role of doers, as they still need to play a role in making the technology work. But information systems professionals should not try to initiate the innovative processes and practices that are responsible for the benefit creation and delivery[16]. Also the information systems professionals should not be responsible for identifying benefits or justifying the expenditure on the systems. The business case for information systems development needs to be created by the line manger, maybe with the help of the other principal stakeholders. who will use the system to improve his or her personal or group efficiency and effectiveness.

11.     Summary and conclusions

The way forward in reducing the degree to which IT investment benefits have been elusive requires the recognition that IT investment needs to be an integral part of a greater program of process innovation and improvement. The success of the IT investment is then tightly coupled with the success of that process innovation and improvement. In fact, the IT investment cannot be assessed or evaluated independently of that process innovation.

If the process innovation and improvement was to improve productivity then the IT investment will be judged on whether that has been achieved. On the other hand, if the process innovation and improvement was to increase customer satisfaction then the IT investment needs be judged on whether that has been achieved.

It is also necessary to accept that some IT investment benefits cannot be satisfactorily stated in monetary terms. A different type of business rationality is involved. Nonetheless they are real business benefits and need to be taken into account in any development of any investment evaluation equation.

Having established the process innovation and improvement element of an IT investment it then follows that IT professionals may not be the most appropriate individuals to initiate or lead such projects. The principal stakeholders of these IT enabled or even IT driven process improvements need to be a P&L manager who is at the same time the information system user-owner. Perhaps more than any other factor the success or failure of the IT investment,  rests with the skill and commitment of the information systems principal stakeholders.

It is important to recognise the fact that IT investment benefits have been difficult to identify because of the issue of information systems reach, the nature of tangible versus intangible benefits, and the question of benefit evolution. A process approach is required to both the delivery and the assessment of  IT benefits.

Finally the locus of responsibility issue needs to be clearly focused if IT benefits are to be identified and evaluated.

Of course, the work in developing a comprehensive approach to  the identification and the management of IT investment benefit delivery is not complete and is unlikely ever to be so. In the words of Checkland (1986) writing about systems thinking:

‘Obviously the work is not finished, and can never be finished. There are no absolute positions to be reached in the attempt by men to understand the world in which they find themselves: new experience may in the future refute present conjectures. So the work itself must be regarded as an on-going system of a particular kind: A learning system which will continue to develop ideas, to test them out in practice, and to learn from the experience gained.’

However it is possible to say that some considerable progress has been achieved in understanding the issues involved with the identification, the evaluation  and the  management of IT investment benefits and that some frameworks have been developed which can improve the ability of organisations to management IT investment benefits.

References

  • Barua, A., C. H. Kriebel and T. Mukhopadhyay (1995) Information technologies and business value: An analytical and empirical investigation, Information Systems Research, 6 (1).

  • Brynjolfsson, E, (1993) “The Productivity Paradox of Information Technology: Review and Assessment”, Communications of the ACM, December.

  • Brynjolfsson, E. and L. Hitt (1995) Information technology as a factor of production: The role of differences among firms, Economics of Innovation and New Technology, 3 (4).

  • Butler M, cited in Lincoln T, (1990), “Managing Information Systems for Profit”, John Wiley and Sons, Chichester,.

  • Currie W, (1989),  “The art of justifying new technology to top management”, Omega, No 17, 5 pp 409-418.

  • Checkland, P., “Systems Thinking, Systems Practice”, John Wiley and Sons, UK, 1986.

  • Davenport, T.H. (1997), Information Ecology. Pg. 3, Oxford University Press, US.

  • Deckle, J., (1986), “The Corporate Integration of Data Processing”, Business, April-June.

  • Earl, M, (1992), Putting IT in its place: a polemic for the nineties, Journal of Information systems, September.

  • Evans C, (1981), The Making of the Micro – A History of the Computer, Victor Gollancz, London.

  • Farbey B, Land F and Targett D, (1995), A taxonomy of information systems application: the benefits’ evaluation ladder, European Journal of Information Systems, No 4.

  • Fukuyama F, (1995) Trust: The Social Virtues and the Creation of Prosperity, Hamish Hamilton, London.

  • Grindley, K., (1984), “Managing IT At Board Townsend, R., “Further Up the Organisation”, Knopf, New York,

  • Hammer, M, Champy, J (1993): Reengineering the Corporation. Nicholas Brealey, London.

  • Handy C, (1998) The Hungary Spirit, Random Hopuse, London.

  • House, E. R., (1993) “Professional Evaluation: Social Impact and Political Consequences”, Sage Publications, Inc.,.

  • Irani, Z. and Patel, N. (1999) Evaluating information technology in dynamic environments: a focus on tailorable information systems.  Logistics Information Management, 12:1/2, pp32-39.

  • Keen, P G W (1991): Shaping the Future: Business Design Through Information Technology. Harvard Business School Press, Boston.

  • Keen, P G W (1997): Process Edge. Harvard Business School Press, Boston.

  • Keidel, R.W., Seeing Organizational Patterns; A New Theory and Language of Organizational Design; Berret Koehler, San Francisco, 1995.

  • Kling R, (1980), Social Analysis of Computing: Theoretical perspectives in Resent Empirical Research, ACM Computing Surveys (12:1), p 61-110.

  • Kumar K, van Dissel H and Bielli P, (1998), The Merchant of Prato – Revisited: Towards a Third Rationality of Information Systems, MIS Quarterly, June p 199-226

  • Lacity, M, and Hirschheim, R, (1995) Information Systems Outsourcing - Myths, Metaphors and Realities, John Wiley and Son, Chichester, England.

  • Lincoln, T., (1990) “Managing Information Systems for Profit”, John Wiley & Sons.

  • McKeen, J.D., H.A. Smith, and M. Parent (1999) An integrative research approach to assess the business value of information technology. In M.A.  Mahmood and E.J. Szewczak (Eds.) Measuring Information Technology Investment Payoff: Contemporary Approaches, Hershey, PA, USA and London, UK: Idea Group Publishing.

  • Mooney, J.G., V. Gurbaxani, and K.L. Kraemer (1995) A process oriented framework for assessing the business value of information technology, Proceedings of the Sixteenth International Conference on Information Systems, December 10-13, Amsterdam, The Netherlands.

  • Nolan R, (1974), Managing the four stages of  EDP growth, Harvard Business Review.

  • Nolan R, (1973), The plight of the EDP Manager, Harvard Business Review, May-Jun.

  • Palmer, C., (1990) “A Bridge So Far”, National Computing Centre.

  • Parker, M. M. and Benson, R. J., (1987) "Information Economics: An Introduction", Datamation, December.

  • Remenyi, D  (1999), IT Investment making a business case, Butterworth Heinemann, Oxford.

  • Remenyi, D, Money A, and Twite, A. (1998), Effective Measurement & Management of IT Costs & Benefits, (Second Printing, Second Edition) Butterworth-Heinemann, Oxford, England.

  • Remenyi D, Sherwood-Smith M, White T, (1997), Achieving Maximum Benefits from your information Systems, Wiley and Sons Chichester, 1997.

  • Scriven, M. S., (1991) “The Science of Valuing”, in Foundations of “Program Evaluation: Theories and Practice”, by Shadish W. R. et al. .

  • Selig, F. F., (1991) “Managing information technology in the Nineties”, Information and Management,.

  • Sherwood-Smith, M. H. (1989) "The Evaluation of Computer-Based Office Systems". Ph.D. Thesis. University College Dublin.

  • Smithson, S. and Hirschman, R. (1998) A critical analysis of information systems evaluation. In Information System Assessment: Issues and Challenges (Bjorn–Andersen N and Davis G. B, Eds), PP 17-37, N. Holland, Amsterdam

  • Soh, C. and L. Markus (1995) How IT creates business value: A process theory synthesis, Proceedings of the Sixteenth International Conference on Information Systems, December 10-13, Amsterdam, The Netherlands.

  • Solow, R.M.  (1987).  Review of S.S. Cohen  and J Zysman  "Manufacturing matters: The myth of the post-industrial economy", New York book review, 12th July.

  • Strassman, P. A., (1985) “Information Payoff: The Transformation of Work in the Electronic Age”, Free Press.

  • Strassmann, P A (1990): The Business Value of Computers. Information Economics Press, New Canaan.

  • Strassmann, P.A. (1997) The Squandered Computer, New Canaan, CT:The Information Economics Press.

  • Symons V, (1991) A review of information systems evaluation: Content, Context and Process, European Journal of Information Systems, No1,3 pp 205-212.

  • Townsend, R., (1984), “Further Up the Organisation”, Knopf, New York.

  • Walsham G, (1993), Interpreting Information Systems In organisations, Wiley, Chichester.

  • Weill, P. (1990) Do Computers Really Pay Off? Washington, D.C.: ICIT Press.

  • Willcocks L and Lester S, (1993) How do organisations evaluate and control information systems investments? Recent UK survey evidence, In Avison D Kendall J and DeGross J, eds., Human, Organisational and social Dimensions of Information Systems Development, North Holland Amsterdam.

  • Willcocks, L, Lester, S (1997): Assessing IT Productivity: Any Way Out of the Labyrinth? In L Willcocks, D Feeny & G Islei (eds.): Managing IT as a Strategic Resource. McGraw-Hill, London, 64-93.

  • Willcocks, L, Lester, S (1999): In Search of Information Technology Productivity: Assessment Issues. In L Willcocks & S Lester (eds.): Beyond the IT Productivity Paradox. Wiley, Chichester, 69-97.

Biographical details of author

Dan Remenyi is a Visiting Professor at Brunel University and a management consultant based in the United Kingdom working worldwide. He is also an associate member of faculty at Henley Management College.

Professor Remenyi started his education in economics and political science by obtaining a B Soc Sc before going into business. After a few years business experience, he completed a masters of business administration (MBA) and then joined the information systems industry and worked in a number of different capacities. Eventually he returned to academe, reading for a doctorate in management (PhD) and completing the degree in 1990.

Since obtaining his PhD, Dan Remenyi has specialised in the area of the formulation and the implementation of process improvement strategies supported by information systems and how to evaluate the performance of these and other systems. In addition, he is involved in research and consulting in matters related to knowledge enhanced process management and creating knowledge oriented organisations. He has had published a number of books in information systems management. His most recent books on information systems management, published by John Wiley and Butterworth Heinemann respectively are called Achieving Maximum Value from Information Systems - A Process Approach, and Stop IT Project Failure through Risk Management and IT Investment Making a Business Case. These books introduce a new approach to managing the use and implementation of complex transformation type projects. He regularly conducts management seminars and briefings for both senior and operational management on how to use the information systems resource to make their organisations more efficient and effective, as well as to give themselves a competitive advantage.

Dr Dan Remenyi has spent more than 25 years working as a management consultant in areas related to the effective implementation of corporate performance improvements based on information technology. His work in this field began during his time with IBM, with Barclays Bank and Barclays Merchant Bank and with the management consultants, Turquand Young and Layton-Bennett (subsequently merged and eventually became part of Ernst and Young). In recent years he has worked for clients such as Mars Confectionery, FI Group, Caterpillar Division of Barlows, Andersen Consulting, National Health Services in the United Kingdom, Ernst & Young, Spoornet (the national railroad company of South Africa), Liberty Life Insurance Company, Anglo-Vaal Mining Corporation, ABSA and First National Bank.

 

[1] There is an important question here as to what constitutes a satisfactorily statement of benefits in monetary terms. By the use of a series of assumptions it is of course possible to produce financial estimates of the impact of even the most intangible benefits.

[2] This term economics of information should not be confused with Information Economics which is a specific concept which has been derived by Marilyn Parker and colleagues (1987).

[3] The primary rationality underpinning business and management phenomenon is sometimes described as being technical-economic (Kumar et al 1998). This conceptual approach focuses on  profit and return on investment and how technology can improve these business performance measurements. However this is not the only conceptual rationality with which to understand business and management especially within the information systems or information technology domain. As Kling (1980) points out there are two distinct information systems rationalities one of which is to do with efficiency and effectiveness while the second is to do with corporate political realities which Lincoln refers to here.

[4] It would not be true to say that IS academics or even consultants have ignored the challenges offered by this demanding area. However although much work has been done and hundreds of academic papers published there has been very little attention paid to attempting to consolidate this work into a coherent theory.

[5] It is easy enough to identify the start of business computing with the introduction of computers to the Bureau of the Census in the United States of America by Eckert and Mauchly and to the J Lyons Organisation in the United Kingdom by Cambridge University in 1952 (Evans 1981; Earl  1988). It is more difficult to be specific about the roots of information systems management. Sometimes it is argued that the first semblance of information systems management may be recognised in the work of Dick Nolan published in the Harvard Business Review in 1973 and 1974. Thus this represents a 20 year lag between the time computers were introduced until they became of serious object of management study. But even today many IT executives will say that their organisations still perceive them to be “techies”, although they have actually become in their day to day work quite far removed from the detail of how the technology works. As was recently pointed out to the authors in a discussion with a senior business consultant, some IT executives hardly know more than how to switch on and off their personal computers and some of them are actually proud of this.

[6] When it come to really complex systems such as ERP then this problem become even more tricky and benefits have to be, in a sense, discovered as the implementation progresses..

[7] There are several different definitions of tangible and intangible benefits. For the purposes of this paper a tangible benefit is one which affects the organisations bottom line and an intangible benefit does not.

[8] This notion of IT investment benefit evolution is a central part of a framework called Active Benefit Realisation which has been described by Remenyi, Sherwood-Smith and White (1998).

[9] A P&L (profit and Loss) person is an economically rational actor (Fukuyama 1995) and who has corporate responsibility to make all or part of the organisation’s profit. This role is contrasted with a staff or specialist person who does not has the responsibility to make profit such as a personnel manger or an information systems manger.

[10] An important by-product of this view of the role of IT in an organisation is the fact that to measure its performance it is not necessary to create any IT specific metrics. General business performance metrics are perfectly adequate as measures of the success or failure of IT investment.

[11] Off course it is actually possible that some extraordinarily eccentric and very wealthy individuals might actually collect bulldozers and in such a case a bulldozer would indeed become in classical economic terms a consumer durable good.

[12] For some organisations there are several issues involved here including the fact that traditionally purchased hardware has been treated as an asset and thus capitalised. Sometimes purchased software was also handled from an accounting point of view as a capitalisable item. On the other hand software which was produced internally was seldom treated as an asset and thus it was quite rare to see that it was capitalised in the accounts. To make this issue even more complex Keen (1991) points out that the value of an organisations data may be worth just as much as all the hardware and software together. Maybe in the past and even the present too much has been made of the distinction between assets and cost and investment and expenditure.

[13] It is perhaps important to note that information systems are not in any way restricted to line or P&L positions within the organisation but are used in all sorts of functions and processes including staff or support activities. However the value arguments will be more easily seen in functions where the organisation’s profit is at stake.

[14] Svendsen (1998) provides a general definition of the word stakeholder as follows: The term 'stakeholders' refers to the individual or group who can affect or be affected by a corporation’s activities.  In the information systems environment the stakeholders are all those individuals and groups who can affect or be affected by the information system.

[15] The idea of locus of responsibility is similar to that of IT governance. The locus of  responsibility would include issues such as how money is allocated to investment, who can give the go ahead for IT investment and who are able to comment on the success of these investments.

[16] Although it is correct to say that IS professionals can be quite imaginative and therefore should be free to suggest process improvements, the point  here is that they should not be sponsors or executive champions of process improvement. This has to be exclusively in hands of P&L people.

Copyright   © Dan Remenyi, 1999

Home Up Papers in this Issue Previous Issues Site Map

EJISE is published by Academic Conferences Limited
Curtis Farm, Kidmore End, Nr Reading RG4 9AY, England
Tel: +44 (0)1189 724148, Fax: +44 (0)1189 724691, Email: info@ejise.com

Send mail to info@academic-conferences.com with questions or comments about this web site.
Copyright © 2002-2006 Electronic Journal of Information Systems Evaluation
Last modified: November 25, 2004
ISSN 1566-6379