|
Too many methods ?
The Problem of Value
The Nature of Value
A Taxonomy of Techniques
An Understanding of Complex
Decision Making
Summary and Conclusion
References
1. Too Many Methods?
The evaluation of Information Systems is one of the more intractable
problems in IT (Smithson and Hirschheim 1988) and, despite much research, particularly
over the past 15 years, (Keen 1985; Hirschheim and Smithson 1988; Currie 1989;
Sherwood-Smith 1989; Scriven 1991:a&b; Symons 1991; House 1993; Walsham 1993;
Willcocks and Lester 1993; Farbey et al. 1995; Remenyi et all, 1991, 1994, 1998,
1999) it continues to defy clear answers or yield up a satisfying overall theory.
IT evaluation is arguably one of the most researched and written about
topics in the IS literature. Apart from being vast (Powell 1992, Banker et al 1993)
the literature is replete with innovative attempts to surmount the theoretical and
practical problems of IT evaluation. Starting from conventional financial and economic
evaluation techniques, researchers have extended the range of tools to include
productivity measures (Brynjolfsson, Hitt, Roach and others) return on management
(Strassmann and others) and information economics (Parker and Benson) to name but some.
Various taxonomies of methods have been put forward by a number of authors including Irani
(1999), Bacon (1994), Lucas (1993) and Cronk and Fitzgerald (1998).
Notwithstanding all of this effort to come up with a rigorous and
replicable ‘toolset’ of methods, several researchers have found that, when
pushed, decision makers, both individual and corporate, often describe their decisions as
being based to a greater or less extent on instinct. Indeed, the more complex the
decision, the more likely this seems to be. For the scientist, such defection from the
solid ground of rational positivist decision making is, at first sight anyway, disturbing.
A variety of terms is used to describe this decision making process, for example
"acts of faith" (Farbey et al. 1993, Deitz and Renkema, 1995), "blind
faith" (Weill 1990) and "gut instinct" (Powell 1992, Katz 1993).
It will be argued in this paper that one of the sources of the gap
between theory and practice is that, in much current research, the concept of value is ill
defined. Indeed it is often absent from the discussion entirely like some axiomatic
definition that all readers are assumed to share. On other occasions, value is defined in
a narrow and/or partisan manner. The difficulty here is that if the user does not accept
or share this worldview, any evaluation technique which uses it as a foundation crumbles.
What is needed is a better understanding of IT value and how users, individual or
corporate assess such value in their own minds.
2. The Problem of Value
Considering the scale of research into IT evaluation, the volume of
research into IT value as such is remarkably small. As noted above, many papers on IT
evaluation either assume that there is a common understanding of the concept, or define it
in such a way as to serve a specific purpose. This can be illustrated by looking at the
Economists’ worldview. In a widely cited paper, Brynjolfsson makes the following
fairly strong statement: "Productivity is the fundamental economic measure
of a technology’s contribution" (Brynjolfsson 1993, p76 emphasis added). The
question that immediately springs to mind here is contribution to what? Firm
profitability? Probably. Customer satisfaction? Maybe. Better decision making? Hardly. To
take another example, one of the best known public sceptics of IT value, the banker and
economist Stephen Roach, described the productivity gains of the computer age as "just
a myth" (Financial Times, 13th August 1997). [Aside: This may (or may
not) be true. The current seven year long non inflationary boom in the USA has one group
of economists crediting IT with big productivity improvements while others declare it to
be an illusion. For an interesting analysis of this issue see Gordon (1999a, 1999b) and
Jorgenson and Stiroh, (1999) [Measurement of productivity is complex.
Gordon (1999a/b) argues that much of the productivity improvement in the 1980s and 1990s
comes from improvements in the manufacture of computers but that the productivity
improvements derived from computer usage shrink into insignificance when compared with
19th century technological developments such as electricity and combustion engine].
The phenomenon has also been commented on by Loveman (1992) who stated that: "Despite
years of technological improvements and investment there is not yet any evidence that
information technology is improving productivity or other measures of business performance
on a large scale - or, more importantly, significantly enhancing US economic performance."
Now simple observation suggests that there has to be something
profoundly wrong with Brynjolfsson, Roach and Loveman’s diagnoses. Investment in IT
over the past 30 years has been enormous and shows no appreciable sign of slowing. IDC
recently reported an estimated annual global expenditure of over $1.8
trillion on IT in 1997. While IT expenditure and value for money continue to be major
management concerns (Price Waterhouse 1994, 1996; Wentworth 1996), the same managers
continue to spend very large amounts of treasure on IT/IS. Why? It the economists are
right, this an act of collective ineptitude on a massive scale. Few, however, would argue
that so many managers and organisations are so irrational.
The steady stream of strong criticism of IT value in the business press
(Lincoln (1990), Earl (1992)) has generated a counter reaction from IT professionals and
IS researchers, although probably the latter have been more active in trying to refute the
claims of poor value – the professionals are too busy making money. Farbey et al
(1993) describe this surge of IS research activity as the "great benefits hunt".
This has taken two forms: the first might be broadly described as econometric and searches
for relationships between input and output data (Loveman (1991), Brynjolfsson (1993, 1994
(with Hitt)) and Hitt and Brynjolfsson (1996)). This branch of research activity rotates
around the so called productivity paradox and where paradox has been lost and even
regained [thank you mr. John Milton-What would they have done without your
Paradise Lost]. Barua et al (1995), Mooney et al (1995)) and others have
developed a process view of IT value. This is a more complex and subtle view of the
evolution of IT value, but in essence it is based on the same or similar metrics. Soh and
Markus (1995) reviewed and endeavoured to synthesise a number of such methods. Treating
value creation as a process provides a richer description of the nature of IT value
evolution, but the underlying metrics used (e.g. ‘improved organisational
effectiveness’), are still essentially economic. To compound the limitations of many
of these approaches, there is a shortage of suitable data. What data there is is limited
and not without problems (Brynjolfssohn 1993). Furthermore, the appropriateness of the
methods used to analyse these data is debatable, the theoretical models open to challenge
and the conclusions either weak, contradictory and/or, most critically, intuitively
suspect. Nevertheless the question persists: if the return on IT investments is so
uncertain, why do organisations continue to invest such vast sums in this technology?
One possible answer to this is that IT is valued in more complex and
subtle ways than the raw economic and financial data suggest. The business and human
concept of value may be much deeper and wider than the narrow rationalism that economic
and accounting models allow. Glazer (1993) pointed this out when he said "...
managers themselves are the best judges of the value of the variables with which they work."
If Glazer is correct, then the proponents of the methods described above may simply be
looking in the wrong place.
Walsham, Farbey , Symons and others put together a more qualitative
view of IT value. Their approach places much more emphasis on the human and sociological
values of Information Systems. It uses a much wider definition of benefits, placing value
in a broader context than accounting and economics, taking into account both hard and soft
or intangible benefits (Symons 1990, Symons and Walshman 1991, Remenyi et al 1994, Symons
1994, Peters 1994). Symons uses the term ‘multiple perspectives’ to emphasise
the location of IS evaluation in social/organisation contexts. This, in turn, implies a
different understanding of value (whether this is implicit or explicit). The question of
what constitutes value is therefore worth reviewing briefly.
3. The Nature of Value
"First and foremost, what ultimately matters is value - to the
firm, individuals, or society" (Loveman 1992, p101). But what is
‘value’? In much of the literature on IT evaluation, there is no discussion of
the concept of value per se. The meaning of the term is assumed to be implicitly
understood. But absence of a clear conception of value can lead to misconceptions about
how useful the measurements used to asses it are. Business value can be ambiguous- even
misleading. Cross (1997) shows how increases in market share and capacity utilisation,
both of which one might instinctively consider to be ‘benefits’, can in quite
common circumstances, be accompanied by a drop in both absolute profit and/or
profitability. Keen (1991) and Strassmann (1997) have examined in some detail the
ambiguities in and contradictions between traditional measures of business value. Even
where value is formally defined there is a broad range of definitions used. For example,
De Rose (1991) defines value as perceived by customers as:
"… the satisfaction of purchase requirements at the lowest total cost in use."
But value can go well beyond such narrow confines. It may, for example, regarded as a
measure of the organisation’s effectiveness. Accountants use the concept of
‘monetary measurement’ (Sidebotham 1970) which posits money as a common
denominator for comparing value. Sidebotham states that for the accountant: "In
general, value means historic cost to the accounting units;" One would expect
that if an unambiguous definition of value was to be found anywhere it would be in
accountancy. But the words ‘in general’ in this definition suggest that even
accountants cannot be unambivalent about the concept (see also Smith (1992)) .
Parker and Benson (1988) base their concept of IT value on
Porter’s value chain (Porter 1985). Value, in their definition, may be summarised as
the ability of IT to enhance the business performance of the enterprise. Wiseman (1992),
develops Parker and Benson's ideas by differentiating between value and benefits,
asserting that value is both larger and more important than benefits. He argues that, for
example, users can develop a strong attachment to an old system. It can thus acquire a
sort of value, despite the fact that it may be out of date and inefficient (although one
could argue that it is still delivering ‘benefit’ in the sense of user comfort).
Berghout and Renkema (1997) define value as the outcome of financial and non financial
consequences of the IT investment – a flexible definition, but one which still leaves
the fundamental nature of value untouched.
Keen (1991, p162) summarises the problem thus: "Many a scholar,
consultant and practitioner has tried to devise a reliable approach to measuring the
business value of IT at the level of the firm, none has succeeded". Keen draws an
analogy with research and development (R&D) arguing that, like R&D, IT does not
create benefits as such. As with R&D benefits, IT benefits are the result of complex
effects with time lags which can be lengthy. This point of view is supported by Nowak
(1991). Nowak argues that there are well established techniques for R&D evaluation
which can be applied to MIS evaluation.
Finally, this topic is also plagued by conflicting evidence. Huff
(1990, p43) states that: "While the findings have not been totally consistent,
generally market leaders have been shown to have invested more heavily in IT as a
percentage of sales, than have average performers". However the Kobler Unit
(1987) albeit looking only at the UK market, and using a small sample, found that there
was no difference in IT spend between market leaders and laggers. What is one to
believe?
Contributions to this debate have also been made in a number of papers
by Cronk and Fitzgerald (Cronk and Fitzgerald 1997, 1997a, 1998). Cronk and Fitzgerald
address the concept of IT business value which they examine in depth. The problem with IT
business value is that is has never been adequately defined. Cronk and Fitzgerald’s
set out to establish an IT business value construct by introducing the concept of
dimensions of value. They suggest three basic dimensions: system dependent, user dependent
and business dependent, which they claim are uncorrelated, and what they term a moderating
contextual dimension. While Cronk and Fitzgerald have added new insights, there are still
problems with their model not least the assumption that their basic dimensions are
uncorrelated – a precept that seems intuitively suspect.
We may safely conclude that the definition of value is far from
universally agreed - especially among information systems academics. The word
‘value’ is, as Veryard (1991, p3) puts it, "nicely ambiguous"
[Classical
economics states that there arre two types of value: value in exchange and value in use.
Accounting is based on value in exchange where the amount of money for which a product
changes hands it its value. Although only of limited use, this notion has the great
advantage of being clear and simple to understand and apply. Value in use is more complex
to understand and has considerable problems in being quantified. The problem which arises
in IS evaluation is that value in exchange is not of much help in assessing the success on
an investment where there is no exchange. We are therefore forced to come to terms with
the quantification of the value in use concept and this is where the problems really
start]. It is not therefore surprising that, in seeking value or benefits,
many techniques have been tried nor that the findings are sometimes contradictory and the
subject of fierce debate.
4. A Taxonomy of Techniques
Investment decisions are based on human perceptions of value, however
measured. An understanding of how value translates to decisions can be aided by
classifying methods of evaluating IT investments into three basic techniques which may be
used in two different ways.
The first level consists of the underlying approaches to evaluation
which we may term Fundamental, Composite and Meta methods.
- Fundamental measures are metrics which attempt to parameterise some
characteristic or closely related set of characteristics of the investment down to a
single measure [The term 'partisan' might also be used here, in the sense
that such measures often take a particular perspective on value. However the term
'fundamental' is probably more meaningful in this context.] The key
characteristic of such methods is that they provide a single score or statistic which is
used to assess an investment. Measures of this type are not confined to the purely
financial, although financial measures are the most common.
- Composite
approaches try to combine several fundamental measures to get a
‘balanced’ overall picture of value and/or investment return. Composite measures
include the Information Economics of Parker and Benson (1988), portfolio methods, the
Balanced Scorecard of Kaplan and Norton (1996), BSC (Ward 1994) and SMART (Goodwin and
Wright 1998). Few organisations which would try to evaluate their information systems
activity today or try to choose between competing projects without using some variant of
the composite approach [Many of these methods are themselves composite in
nature. For example, even a net present value is derived from net cash flows which in turn
are derived from a range of other factors and computations. Cost benefit analysis is also
based on using money as a metric for combining many factors, some of which are distinctly
non-monetary in origin. This repertoire of techniques had evolved into a shopping list of
methods which are applied in day to day IT investment evaluation. A brief scan of the
literature will reveal numerous cases where these methods are cited or where researchers
have endeavoured to find out which of these methods are used in practice].
- Meta approaches (e.g. Farbey et al, 1993, Peters 1994) attempt to select the best
set of measures for a context or given decision. This meta orientation is generally not
structured. Each case will be different and there is no question of the organisation
wishing to use this approach for any sort of benchmarking.
These three approaches may be applied in two different ways:
- Positivist or Reductionist where the decision maker allows (one might
almost say empowers) the methodology to make the decision. The decision maker establishes
a series of mechanical (and replicable) operations which often reduce the decision to a
single score. This can be done using any of the three fundamental approaches.
- Hermeneutic, here defined as methods of interpretation of data which use
non-structured and non formal approaches to both understanding and decision making. Using
this approach the decision maker takes on board several different metrics directly and
combines them in his or her mind in a manner that is, at best, extremely difficult to
describe formally. When decisions are made this way, instinct and intuition generally play
a major role.
This concept is illustrated in figure 1.
Figure 1 Three techniques and two approaches to IT Evaluation
In reviewing the literature, it is clear that much of the current
research into IT evaluation is focused on the positivist at the expense of what to us is
the more interesting, but much more difficult to confront, hermeneutic. We argue not only
that hermeneutics is pervasive, but that there are limits to the purely positivist
approach. The number of evaluation methodologies and approaches is finite, but the number
of possible IT investment/expenditure situations is potentially infinite. It follows that
there is an infinite number of possible mappings from the set of evaluation techniques
onto what we might term the investment opportunity space. Clearly some subset of the
investment opportunity space can be meaningfully evaluated using an appropriate single
basic measure of value. However outside of this subset, an immediate tension between
positivism and hermeneutics arises. This can be seen in what the composite and meta
methods set out to achieve:
- Composite (and, to a lesser extent, meta) approaches try to model how some sort of ideal
decision maker should (or would want to) evaluate an investment. Formalised composite
methods, such as Information Economics, are tantamount to a semi prescriptive statement
that this is the way an IT investment decision should be made. Given that we have
an infinite number of decision making situations, this seems to us to be highly
presumptive. At best, composite methods provide a substitute for hermeneutics and an
implicit recognition of the need for interpretation.
- The meta approach attempts to match decision problem to ‘best of breed’
technique. Meta methodologies can be useful in helping to identify the most appropriate
evaluation technique. The weakness in the meta approach is the necessity to try to derive
general rules for a continuum of decision making situations from a finite set of tools.
Inevitably, the conclusion of this process will often be tantamount to saying ‘there
is no perfect tool, but this is about as good as we can get’.
All formal methodologies and combinations of methodologies are
constrained by limits of physical representation by numbers, two dimensional diagrams and
the boundaries of modelling human reason using such tools. It may be that the closest we
can get to actual workings of the managerial mind in complex situations is to use such
tools as Likert scales, cognitive maps and spider charts (see Remenyi et al (1995) for a
compilation of such techniques).
5. An Understanding of
Complex Decision Making
Our concern here is instinct, a method of making decisions which occurs
in that subset of the evaluation techniques or approaches which employ hermeneutic methods
and which cannot be easily handled by the traditional approaches. Ultimately,
irrespective of whether a composite approach or a meta approach has been taken to the IT
investment analysis, eventually a decision has to be made and this means some form of
hermeneutic assessment. No matter how quantitative the analysis has been some person will
soon or later have to make a judgement, and this is the most difficult part to comprehend
(Collins 1994).
In order to move towards an understanding of this process we can look
at other fields of complex decision making. Anybody who has witnessed or participated in
an argument on suicide for example, will be aware many people, faced with such a complex
issue, one which has moral, religious, social, medical, political and economic overtones
arrive at a stance that says something like: "suicide is wrong, but there are
circumstances where it can be justified". If confronted with this position and asked
to justify it in logical (positivist) terms, they often cannot do so. This type of
difficult decision making or judgement is frequently made without going through the
apparent rational step by step processes which management decision makers are expected to
follow. This process of internal cogitation and conclusion which is fed by instinct and
intuition has been discussed by many philosophers including Aristotle who differentiated
between what he termed techne and phronesis, what might today we might call
technical and practical reasoning (Dunne 1993). The process of practical reasoning was
described vividly by Newman who called it the ‘Illative Sense’ (Newman 1889).
Other more recent defenders of phronesis include the German philosophers Gadamer (1989)
and Habermas (1984).
Although, in theory, one would normally expect to be able to
differentiate between the value of an IT investment to an organisation and its values to
the decision makers, in practice, in the mind of the person or persons making the
decision, both are confounded. We all bring what Gadamar calls
our own prejudices to the decision. Our decisions are influenced not only by the
‘objective’ analysis of numbers, returns, income, costs and so on, but by
cultural, political, personal and other subliminal factors; what Dunne calls the sub-soil
of the psyche. This is what decision makers confront when making complex decisions about
IT investments. They could not easily rationalise this process even if they tried. Instead
they call it instinct, faith, intuition etc.. Instinct is not therefore, necessarily
something to be condemned – an abandonment by the decision maker of reason. Rather it
is often a different and more subtle kind of reasoning – a method of taking into
account how the world really is rather than simply what the spreadsheets say. Its
weakness, of course, is that it can be wrong – but then so can so called rational
decision making when the assumption and information on which it is based is incorrect.
A model of this decision making process might be as shown in Figure 2.
Figure 2: Decision Process Model
Any decision is influenced by a range of factors, some rational, some
non-rational, some explicit others implicit. These factors carry different weights in the
minds of different decision makers. Each is derived from external information of various
types. This, in turn, comes through various filters: subordinates, consultants,
journalists and salespeople and so on. We term these external filters.
This information then goes through a further lens of personal
experience and psychological make-up, before being assimilated and weighed up to make a
decision. We term these internal filters. A large component of the interior filter
is the decision maker’s personal perception of value. Furthermore, most decisions,
while made at a point, mature over time to that point, i.e. there is a moment at which the
decision is made, but there may be a prolonged gestation period, even after the sources of
external information have ceased supplying new data.
6. Summary and Conclusion
Sound management decision making requires a combination of talents.
Rational thinking processes alone are not sufficient. To be successful management decision
making requires at least rationality plus good instinct. It is clear that instinct is a
central part of many decision making processes and especially the management decision
making process. Without an understanding of instinct we will have a very incomplete
understanding of management.
Human decision makers, working alone or in groups within large
organisations, are rarely, if ever, as logically rational as many commentators, including
themselves, would like to believe. Many IT investment decisions, maybe even a majority,
are made or apparently made, and rightly so, on purely technical rational grounds. Such
decisions may be made using the same type of formal structure that might be used to buy a
factory, develop a new product, build a house or play bridge. But much of the time, and
particularly for large and/or complex decisions, the process of evaluating IT is the
application of phronesis, the application the absorption of a range of input
information including data, evaluation techniques, personal experience, personal
knowledge, corporate or departmental politics, personal desires and intuition; a process
of filtration and distillation of frequently very complex data, information and knowledge
to levels manageable to the human mind.
Whether the incorporation of all of these factors is conscious or
unconscious, they are always present. Models which seek to provide surrogates for such
‘irrational’ factors may be employed, but if they conflict with the inner
conviction of the decision maker(s), they may be rejected. The positivist may describe
such a rejection as ‘irrational’, but this view is based on the premise that the
decision maker shares the same values and has access to exactly the same knowledge as the
observer something that is arguably never the case. The disconcerting fact remains that
good business (and other) decisions are sometimes taken in the teeth of the
‘evidence’ [Military history is a good example of such examples. A
particulary good case is US decision making during the battle of Midway (prange et al,
1982)] . It is this capacity to make intuitive leaps that often
distinguishes the great manager from the competent functionary.
This internalised, subjective and idiosyncratic knowledge and knowledge
processing, referred to in this paper as instinct, is an essential part of decision making
and should not be in any way disregarded or denigrated. In fact it is the authors’
suggestion that this instinct should not only be defended but it should actually be
celebrated as part of not only that which differentiates man from machine but separates
mediocre from top flight management.
After all value, like beauty and the contact lens, remains in the eye
of the beholder and the eye of the beholder in business and management situations needs to
be cultivated. Were it any other way, there would be far fewer poor or bad business
decisions - whether IT related or not.
References
-
Bacon, James (1994): "Why companies invest in information
technology" in: Leslie Willcocks (editor), Information Management, The
evaluation of information systems investments, Chapman Hall, London, 31-47.
-
Banker, Rajiv, Robert Kauffman and Mo Adam Mahmood (1993):
"A Comprehensive Bibliography of the Strategic and Economic Value of IT" in:
Rajiv Banker, Robert Kauffman and Mo Adam Mahmood, Strategic Information Technology
Management, Idea Group Publishing, Harrisburg, USA, 607-657.
-
Barua, Anitesh, Charles Kriebel and Tridas Mukhopadhyay (1995):
"Information Technologies and Business Value: An Analytic and Empirical
Investigation", Information Systems Research, 6:1, March, 3-23.
-
Bernstein, P (1996) Against the Gods, John Wiley and Sons, New York
-
Brynjolfsson, Erik (1993): "The Productivity Paradox of
Information Technology: Review and Assessment", Communications of the ACM,
December, 66-77
-
Brynjolfsson, Erik and Lorin Hitt (1994): "Paradox Lost?
Firm-level Evidence on the Returns to Information System Spending", Management
Science, 42, 4, 541-588.
-
Collins, H (1994) A broadcast video on science matters entitled Does
Science Matter?, Open University, London, England.
-
Cronk, Marguerite and Edmond Fitzgerald (1997a), A Conceptual Framework
for Furthering Understanding of ‘IT Business Value’ and its Dimensions, Unpublished
paper, University of Toowoomba, Southern Queensland, Australia
-
Cronk, Marguerite and Edmond Fitzgerald (1997b), "A Contextual
Moderator of Information Systems Business Value" in: Egon Berghout and Dan
Remenyi (editors), Evaluation of Information Technology, Proceedings of the Fourth
European Conference on the Evaluation of Information Technology, Delft University Press,
pp 131-138
-
Cronk, Marguerite and Edmond Fitzgerald (1998), ‘IS Business
Value’: Operationally Defining the Dependent Variable, Unpublished paper,
University of Toowoomba, Southern Queensland, Australia.
-
Cross, Robert (1997): Revenue Management: Hard Core Tactics
for Market Domination, Orion Business Books, London.
-
Currie, W., "The art of justifying new technology to top
management", Omega, Vol. 5, No. 17, pp 409-418, 1989.
-
Davis, S and Davidson, (1993), B 2020 Vision, cited by Tom Peters,
Liberation Management, p 107, Alfred A Knopf, New York.
-
Deitz, Rob and Theo Renkema (1995): "Planning and
justifying investments in information technology: A framework with case study
illustrations" in: Ann Brown and Dan Remenyi (eds.), Proceedings of the
Second European Conference on Information Technology Investment Evaluation, The
Operations Research Society (UK), 28-41.
-
De Rose, Louis J. (1991), Meeting Today’s Buying Influences With
Value Selling, Industrial Marketing Management, 20, pp 87-91.
-
Dunne, Joseph (1993): Back to the Rough Ground,
University of Notre Dame Press, Notre Dame.
-
Earl M, (1992) 'Putting IT in its place: a polemic for the nineties', Journal
of Information Systems, September.
-
Farbey, Barbara, Frank Land and David Targett (1993): IT
Investment. A study of Methods and Practice, Butterworth Heinemann, Oxford.
-
Farbey B Land F and Targett D, A taxonomy of information systems
application: the benefits’ evaluation ladder, European Journal of Information
Systems, No 4, p41-50, 1995.
-
Financial Times (1997): "Freedom Fantasy", Victoria
Griffith, 13 August 1997.
-
Finne H, Levin M & Nilssen T, Trailing Research: A Model for Useful
Program Evaluation, Evaluation Vol. 1, No 1, July 1995 Sage Publications, London, UK
-
Gadamer, Hans-Georg (1989), Truth and Method (Warheit und Methode),
2nd revised edition, Translated by Donald. G. Marshall, Sheed and Ward, London.
-
Garrity, Edward J. and G. Lawrence Sanders (1998): "Introduction
to Information Systems Success Measurement" in: Edward Garrity and Lawrence Sanders
(editors), Information Systems Success Measurement, Idea Group Publishing, Hershey,
PA, 1-5,
-
Glazer R, Measuring the Value of Information: The
Information-Intensive Organisation, BM Systems Journal, Vol32, No1, 1993.
-
Goodwin, Paul and George Wright (1998), Decision Analysis for
Management Judgement, Wiley, London.
-
Gordon, Robert (1999): Has the New Economy rendered the Productivity
Slowdown Obsolete?
http://faculty-web.at.nwu.edu/economics/gordon
-
Gordon, Robert (1999): US Economic Growth Since 1870: One Big Wave?,
The American Economic Review, May 1999.
-
Habermas, Jurgen (1984), The Theory of Communicative Action: Volume
1: Reason and the Rationalization of Society, Translated by Thomas McCarthy,
Cambridge: Polity, UK.
-
Hitt, Lorin and Erik Brynjolfsson (1996):, "Productivity
Business Profitability and Consumer Surplus: Three Different Measures of
Information Technology Value", MIS Quarterly, 20, 2, June
-
House, E. R., "Professional Evaluation: Social Impact and
Political Consequences", Sage Publications, Inc., 1993.
-
Huff, Sid (1990) "Evaluating Investments in Information
Technology", Business Quarterly, Spring, 42-45.
-
Jorgenson, Dale and Kevin Stiroh (1999): Information Technology and
Growth, The American Economic Review, May 1999.
-
Kaplan, R and Norton, D (1996) The Balanced Scorecard, Harvard
Business School Press, Boston, MA.
-
Katz, Adolph (1993): "Measuring Technology’s Business
Value", Information Systems Management, Winter, 33-39
-
Keen, P., "Computer based decision aids: the evaluation
problem", Sloan Management Review, Vol. 16, No. 3, pp 17-29, 1985.
-
Keen, Peter G.W. (1991): Shaping the future : business design
through information technology, Harvard Business School Press, Boston, Mass.
-
Kobler Unit (1987): Does Information Technology Slow You
Down?, Kobler Unit, Imperial College, London.
-
Lincoln, T, (1990) Managing Information Systems for Profit, John
Wiley and Sons, Chichester, England.
-
Loveman, Gary W. (1991): "An Assessment of the Productivity
Impact of Information Technologies" in: Information Technology and the
Corporation of the 90s, Michael S. Scott Morton (editor) , Oxford University Press.
-
Loveman, Gary, Computerworld, November 25, 1991, cited in Tom Peters, Liberation
Management, p 125, Alfred A Knopf, New York, 1992.
-
Lucas, Henry (1993): "The Business Value of Information
Technology: A Historical Perspective and Thoughts for Future Research" in:
Rajiv Banker, Robert Kauffman and Mo Adam Mahmood, Strategic Information Technology
Management, Idea Group Publishing, Harrisburg, USA.
-
Mooney, John, Vijay Gurbaxani and Kenneth Kraemer (1995):
"A Process Oriented Framework for Assessing the Business Value of Information
Technology," in: Janice DeGross, Gad Ariav, Cynthia Beath, Rolf Hoyer and
Chris Kemerer, Proceedings of the Sixteenth Annual International Conference on
Information Systems, Amsterdam, The Netherlands, December 10-13,1995, 17-27.
-
Newman, John (1889), An Essay in Aid of a Grammar of Assent,
1985 Edition by IT Ker, Claredon, Oxford.
-
Nowak, Paul (1991): "Cost-benefit analysis for MIS
expenditure using models developed for justifying research in R&D" in:
Richard Veryard (editor), The Economics of Information Systems and Software,
Butterworth-Heinemann, Oxford, 89-97.
-
Parker, Marlyn and Robert Benson (1988): Information
Economics: Linking Business Performance to Information Technology, Prentice-Hall,
Englewood Cliffs, New Jersey.
-
Peters, Glen (1994): "Evaluating your computer investment
strategy", in: Leslie Willcocks (editor), Information Management, The
evaluation of information systems investments, Chapman Hall, London, 133-150.
-
Porter, Michael (1985): Competitive Advantage, The Free
Press, New York.
-
Powell, Philip (1992): "Information Technology Evaluation:
Is It Different?", Journal of the Operations Research Society, 43, 1, 29-42.
-
Prange, Gordon W., Donald Goldstein and Katherine Dillon (1982): Miracle
at Midway McGraw-Hill, New York.
-
Price Waterhouse (1993/94): International Technology Review,
Price Waterhouse , Menlo Park, California.
-
Price Waterhouse (1995/96): Technology Review, Price Waterhouse
, London.
-
Remenyi, D. and Money, A., "Service Quality and correspondence
analysis in determining problems with the effective use of computer services", European
Journal of Information Systems, Vol. 3, No. 1, pp 2-12, 1994.
-
Remenyi, D, Money A, and Twite, A. (1995), Effective Measurement
& Management of IT Costs & Benefits, Butterworth-Heinemann, Oxford, England.
-
Remenyi, D, Money A, and Twite, A. (1998), Effective Measurement
& Management of IT Costs & Benefits, (Second Printing) Butterworth-Heinemann,
Oxford, England.
-
Remenyi, D, (1999), Stop IT Project Failures Through Risk Management,
Butterworth Heinemann, Oxford, England.
-
Scriven, M. S., "Evaluation thesaurus", 4th
Edition, Newbury Park, CA: Sage, 1991:b
-
Sherwood-Smith, M.H. "The Evaluation of Computer-Based Office
Systems", unpublished PhD Thesis, University College Dublin. 1989.
-
Walsham G., "Interpreting Information Systems in
Organisation", John Wiley, Chichester, 1993.
-
Sidebotham, Roy (1970): Introduction to the Theory and
Context of Accounting, Pergamon Press, London.
-
Smith, Terry (1992): Accounting for Growth: Stripping the
Camouflage from Company Accounts, Century Business, London.
-
Smithson, S. and Hirschman, R. (1998) A critical analysis of
information systems evaluation. In Information System Assessment: Issues and Challenges
(Bjorn- Andersen, N. and Davis G. B. EDS), PP 17-37, N. Holland, Amsterdam.
-
Soh, Christina and M. Lynne Markus (1995): "How IT creates
business process value: A process theory synthesis" in: Janice DeGross,
Gad Ariav, Cynthia Beath, Rolf Hoyer and Chris Kemerer, Proceedings of the Sixteenth
Annual International Conference on Information Systems, Amsterdam, The Netherlands,
December 10-13,1995, 29-41.
-
Strassmann, Paul (1997): The Squandered Computer,
Information Economics Press, New Canaan, Connecticut
-
Sutherland, Frances (1995): "Towards a model for
benchmarking Information Technology effectiveness" in: Ann Brown and Dan
Remenyi, Proceedings of the Second European Conference on Information Technology
Investment Evaluation, The Operations Research Society (UK), 118-132.
-
Symons, Veronica (1990): Evaluation of information systems
investment: towards multiple perspectives, Ph.D. Thesis, Cambridge University,
England, 1990.
-
Symons V, A review of information systems evaluation: Content, Context
and Process, European Journal of Information Systems, No1,3 pp 205-212, 1991.
-
Symons, Veronica and Geoff Walsham (1991): "The evaluation
of information systems: a critique" in: Richard Veryard (editor), The
Economics of Information Systems and Software, Butterworth-Heinemann, Oxford, 71-88.
-
Symons, Veronica (1994): "Evaluation of information systems
investment: towards multiple perspectives" in: Leslie Willcocks
(editor), Information Management, The evaluation of information systems investments,
Chapman Hall, London, 253-268
-
Veryard, Richard (1991): "Introduction: Value and
values" in: Richard Veryard (editor), The Economics of Information Systems
and Software, Butterworth-Heinemann, Oxford, 1-12.
-
Walsham G, Interpreting Information Systems In organisations,
Wiley, Chichester, 1993.
-
Ward, John (1994): "A portfolio approach to evaluating
information systems investment and setting priorities" in: Leslie Willcocks
(editor), Information Management, The evaluation of information systems investments,
Chapman Hall, London, 81-98
-
Weill, Peter. (1990), "Strategic investment in information
technology: An empirical investigation", Information Age, 12, 3, July,
143-147.
-
Wentworth Research (1996), Assessing IS Performance, Research
Report, Wentworth Management Programme, November 1996.
-
Willcocks L. and Lester S. (1993), "How do organisations evaluate
and control information systems investments? Recent UK survey evidence", In Avison D,
Kendall J and DeGross J, eds., Human, Organisational and social Dimensions of
Information Systems Development, North Holland Amsterdam, 1993.
-
Wiseman, Devra (1992): "Information Economics: a
practical approach to valuing information systems", Journal of Information
Technology, 7, 169-176.
About the Authors
Frank Bannister is a Senior Lecturer in Information Systems at
Trinity College, Dublin where he is also Director of Studies of the Management Science and
Information Systems programme. After working for a number of years in Operations Research
in the Irish Civil Service, he spend 16 years as a management consultant with Price
Waterhouse and remains a Consulting Associate with PricewaterhouseCoopers. His research
interests include IT value and evaluation in Public Management/Administration, IT/IS
systems acquisition and investment generally and the use of computer modelling techniques
to aid decision making. He has written many papers and articles on a range of IT and IS
topics and extensively on IT in Ireland and the U.K. He is currently editing an Irish
edition of IT Policies and Procedures, published by GEE. As well as lecturing in Trinity,
he is an external lecturer at the Irish Management Institute and runs specialist courses
for the Irish Civil Service Training Centre.
Dan Remenyi is a Visiting Professor in Information Systems
Management at Brunel University in the United Kingdom and Chalmers University of
Technology in Sweden and a management consultant based in the United Kingdom working world
wide. He has spent more than 25 years working in the field of corporate computers and
information systems. He has worked with computers as an IS professional, business
consultant and as a user. In all these capacities he has been primarily concerned with
benefit realisation and obtaining the maximum value for money from the organisations
information systems investment and effort.
In recent years he has specialised in the area of the formulation and the
implementation of strategic information systems and how to evaluate the performance of
these and other systems. He has also worked extensively in the field of information
systems project management specialising in the area of project risk identification and
management. He has written a number of books and papers in the field of IT value and
management and regularly conducts courses and seminar as well as working as a consultant
in this area. His most recent books published are Achieving Maximum Value from
Information Systems - A Process Approach and Preparing and Evaluating a Business
Case For IT Investment. |
