Information systems provide economic benefit. One way to measure the benefit is by calculating the difference in expected decision payoff with and without information. These calculations can be easily performed using simple decision trees. What needs to be established is the magnitude of economic benefit a manager can expect to obtain from the use of information systems. That benefit amount is related to the accuracy of the information system. Accuracy refers to the system’s ability to correctly predict an occurrence such as predicting the next period demand for a product.

One goal of this research is to provide evidence in support of research which examines the macro economic benefits of information systems and technology. The macro research stands on its own merit but demonstrating economic benefit from the aggregation of individual information systems strengthens its credibility. Also, it provides a bridge between the assertion of information systems in toto having economic value to the specific value for an information system being considered by a manager.

The problem of justifying the expense of individual information systems has a long history. Measuring the dollar impact, when performed at all, generally occurs after the information system has been put into place. Determining the quantifiable dollar amount of an implemented information system is important, but it would be more useful to managers before the information system is undertaken. A method for quantifying the estimated dollar value of a proposed information system aids the manager’s decision to accept or reject a proposed system.

While the above seems obvious, objective measurements of information system value have not been well received. Hogue and Watson [1983] found that only about one in five organizations considered the costs and benefits of an information system before the decision was made to develop that system. Bacon [1992] provides comparisons (Table 5 in the article) showing how an organization uses information system investment criteria. Financial criteria (net present value, internal rate of return, budgetary constraints, and similar measures) as well as managerial criteria (support of implicit or explicit business objectives, response to competitive systems, and others) are surveyed and a higher percentage of companies use managerial criteria than financial criteria.

Bacon’s table provides the percentage of companies using certain criteria as well as the percent of projects in that company to which the criteria were applied. Managerial criteria dominate. The most frequent financial criteria was ‘budgetary constraint’ and yet that criteria was only applied to 43.5% of projects. Economic measures of information system value clearly need to play a larger role.

The investment in information technology dramatically increased from 1970 to today, yet the productivity of white collar workers has been stagnant during this time frame. Organizations are increasingly skeptical about the value of information systems and technology. Brynjolfsson [1993] provides an excellent overview of the concerns. He offers the following four categories of explanations why there has not been documented evidence supporting the contribution of information technology; 1) mismeasurement, 2) lags, 3) redistribution, and 4) mismanagement. Brynjolfsson contends that one facet of the mismanagement category is the lack of cost-benefit analyses of projects.

Loveman [1991] reports government figures showing about half of U.S. expenditures for durable equipment went for information technology equipment. Keen [1991] argues the true investment is much greater because of the organizational costs associated with investments in information technology. Whatever the exact dollar amount spent annually on information systems and technology, it is too large to escape quantifiable estimates of its value.

This article examines the economic payoff organizations can expect from using information systems to support managerial decision making. For example, what is the economic benefit of an information system that can predict the next period sales as stable, increasing, or decreasing? As opposed to the tangible costs of labor versus machine, the economic value of information systems must deal with the intangible, difficult to estimate improvement of the decision. The economic value is calculated as the difference between the expected payoff of decision making with and without the use of the information system.

Decision trees are used to quantify the value of an information system that supports a managerial decision. It is a simple, yet effective tool that managers can understand and implement themselves. The decision tree assesses economic value based on using an imperfect information system, estimates for the dollar impact of decisions, and probabilities for which several conditions could occur.

Simulations offer evidence of information system value that verifies research performed on industry level data. Simulating large numbers of information systems over a range of accuracies yields observations of economic value that can be subjected to statistical analysis. Since the values are generated from ‘the bottom-up’ they provide a contrasting set of data to the industry level data. Information system accuracy, dollar payoff, and other values were varied over a wide but controlled range in order to provide data to be subjugated to hypothesis testing.

The majority of research studying the value of information systems focus on macro issues and base findings on broad economic measures; see Tables 1, 2, and 3 in Brynjolfsson [1993]. While this is good for research in the field of information systems, it is not sufficient for making decisions concerning individual information systems or a portfolio of systems within an organization. Brynjolfsson offers explanations of the productivity paradox of technology as he devotes one section to mismanagement (1993, page 75). An element of that mismanagement is that managers do not apply cost/benefit accounting to technology. Decision trees explicitly determine expected benefits. Such cost/benefit analyses must be applicable to individual information systems. This research uses the value of individual information systems to derive broader conclusions of information system value.

Simulations of 40,000 information systems were generated and analyzed based upon system accuracy, dollar payoff, and other values. The results give some support to critics of information systems who say too much is being spent for too little return. There are times when inaccurate information systems do not change the decision that would be made without the information system; i.e. when one decision action dominates all others. The simulations show that information systems which have a mean accuracy of about 75% will have a dominating action strategy over 25% of the time.

However, even these moderately accurate systems provide an average 22% increase in expected return of decision making compared to decisions made without an information system. This value is derived from the generation of simulations, a ‘bottom-up’ approach to deriving the economic value of information systems. It supports the findings of Brynjolfsson and Hitt [1996, page 542] that "... IS spending has made a substantial and statistically significant contribution to the output of firms". Their research is based on wide economic data for 367 different firms including market value of central processing computer units, total information systems budget, and other data. From either direction of data gathering, bottom-up or top-down, the results show a decided economic value to the use of information systems.

It can be difficult for managers to specify probabilities for information system accuracy that are mathematically feasible. To illustrate this difficulty a survey was given to a group of managers presenting a hypothetical information system. Managers were presented with a scenario where they would have to either accept or reject a shipment of raw materials. They were given the historic probabilities of a raw materials shipment being superior, adequate, or inferior (30%, 60%, and 10% respectively).

The managers were asked to complete a table of conditional probabilities that represented the accuracy of an information system that might predict whether a raw materials shipment would contain superior, adequate, or inferior materials prior to the decision to accept/reject an offered order. An example response is given in Table 1. Interior values represent the probability of an actual raw material quality occurring given a predicted quality for the raw material.

Table 1

Example of Infeasible Probabilities

The Table 1 example is plausible but it represents an infeasible set of conditional probabilities. The prediction probabilities must be independent from the actual/historic probabilities; i.e. a prediction cannot cause an actual quality to occur. Ergo, the values of table cells are calculated as the product of the associated marginal probabilities. The marginal values for actual quality (30%, 60%, and 10%) can be given from past history but the marginal probabilities of the predictions must be calculated.

The conditional probabilities in the table represent the probability of a given raw material quality actually occurring given a predicted quality; P(actual quality | predicted quality). 10% equals the probability of an adequate raw material quality occurring after an inferior quality was predicted. Managers are comfortable expressing the conditional probabilities in the sequence of a prediction being made followed by an observed actual quality. Unfortunately, the conditional probabilities are rendered before the marginal probabilities of predictions are known.

Marginal probabilities for predictions can be obtained by solving the simultaneous equations represented by Table 1 (three equations and three unknowns) with the addition of two constraints. First, the marginal probabilities must sum to one. Second, each marginal probability must be greater than or equal to zero. The formal representation is shown below. Note that P_superior represents probability of predicting the raw material will be superior.

subject to : P_superior + P_adequate + P_inferior = 1

P_superior, P_adequate, P_inferior > 0

Solving these equations provides the values P_superior = .3846, P_adequate = .7372, and P_inferior = -.1218. The P_inferior value violates the second constraint.

In order to gauge managers’ abilities to provide feasible probabilities, 52 managers and professionals were surveyed and asked to provide the probabilities for an information system using the scenario presented above. Only 14 (27%) provided probabilities that were feasible. 37% of the respondents had either initiated or helped design an information system within 12 months of completing the survey. 37% of the survey respondents were female and 63% were male. The average tenure of experience was 7.9 years with 48% of respondents having between 3 and 7 years of experience. 2% of respondents were senior level managers, 38% were middle level managers, 10% line managers, and 50% were professionals.

Of the respondents, 43% reported significant involvement in the initiation and/or design of an information system. Of those, almost half had indicated significant involvement within the last 3 months. The subset of respondents that indicated significant information system involvement were classified according to management level. 6% were senior level managers, 47% were middle level managers, 6% line managers, and 41% were professionals. Management levels were similar among respondents who were and were not significantly involved in the initiation and/or design of information systems.

An analysis of variance was performed to see if there were significant differences in the characteristics of those managers and professionals who provided feasible probabilities from those that did not provide feasible probabilities. Age, gender, highest education level achieved, years of work experience, months since the respondent initiated or helped design an information system, and other factors were analyzed. There were no statistically significant differences between respondents who provided feasible versus infeasible probabilities across any of these factors.

Simulations were used to generate sets of feasible conditional probabilities for this research to avoid the problems of probabilities provided by managers and professionals. 40,000 randomly generated feasible conditional probabilities were produced. Use of simulations for the analysis is important for two reasons. First, values used to calculate the economic benefits of information systems are mathematically consistent. Second, evidence will be generated from a bottom-up approach to validate the findings of Brynjolfsson and others that dollar benefits accrue from the use of information systems.

Information system value in this research is determined by the change in expected dollar payoff from decision making without the information system versus using the proposed information system. There are a number of articles written that focus on establishing the economic benefits of an information system based on results after its implementation [Schumann, 1989, Due, 1996, Chandler, 1982, Mukhopadhyay, et al, 1996] . This is a necessary audit to insure information systems provide benefits that they promise. But managers need a realistic expectation of those benefits before the decision is made to develop the information system. Measurement after the fact may only establish the magnitude of a failed system that should not have been developed.

Van der Schouten, et al [1994] provide a specific example of estimating the value of an information system. The example is quantified by inventory holding costs savings due to the exchange of information concerning the status of production runs. Mukhopadhyay, et al [1995] study electronic data transfer at Chrysler Corporation. Belcher and Watson [1993] look at Conoco, Incorporated’s valuation of an executive information system. The article notes that the ".. decision to develop an EIS is usually based upon executive mandate rather than on an analysis that shows that the benefits will outweigh the costs.." (page 240) and it provides a review of literature concerning what should be measured and how.

The above articles bring quantitative measurements to specific information systems. However, they do not provide evidence on the benefits of information systems in general. A methodology for determining economic value is needed that can be applied to decision support systems.

This article uses decision trees as a method for establishing the expected dollar value of decision support systems with certain characteristics. The first characteristic is that the manager should be faced with a decision where the choice set for actions is clearly defined. Second, knowing that information systems are fallible, the manager must be able to assign probabilities to the information system’s accuracy. Last, the manager must assign a dollar payoff to decisions made. These characteristics are common in decision support systems.

Why use decision trees? They are relatively simplistic yet capture several aspects of decision making. Decision trees capture the sequence of choosing whether or not to use an information system, then observing information system predictions, choosing an appropriate action, and finally observing whether or not the prediction comes true. Most managers can use a spreadsheet package to perform the decision tree calculations. Expected payoffs and other values can be changed to perform sensitivity analysis on the model. Decision tree analysis is a good tool for managers. Each decision tree analysis used in this research became an observation upon which an analysis of information system accuracy and economic benefit could be made.

MBA students were assigned the task of building spreadsheets similar to the example spreadsheet above. The students charged with the task were all managers or professional staff. Individuals developing the spreadsheets required two to three hours. Teams of students (two, three, and four member teams) required one to two hours. The additional cost and effort required to develop a decision tree is very small in relation to the benefits it can provide.

The increase in expected payoff for decision making with the information system versus decision making without the information system represents the greatest amount the user should pay to develop the information system. A novice might be tempted to believe that the expected payoff when using an information system can be completely attributed to the information system itself. But an information system has value only in the change of expected payoff for decision making versus not using an information system.

This example illustrates how a manager would use a decision tree to generate the expected economic benefits for using an information system. In this research, 40,000 simulated decision trees were generated to provide data for statistical analysis. This example provides instruction as well as the rationale for using decision tree analysis to determine the economic benefit for using an information system.

Assume that an information system is being considered that will estimate demand levels for a set of products over the next planning period. The organization assumes certain production levels of each product during that time frame and commits to the purchase of resources to meet production. The organization is capable of some flexibility in manufacture of the products but miscalculating product demand will affect revenues. Estimation of product demand is therefore important to the decision concerning manufacturing levels in this example.

In this scenario, the manager can review product demand history to determine probabilities relevant to demand. Decision actions to be taken and their respective payoffs can also be estimated. Using the expected payoffs of actions and the history of actual demand the manager can calculate the expected payoff of each decision. The action with the highest expected payoff could then be determined.

However, the manager might choose to develop an information system which would predict product demand. The accuracy of such predictions would be a function of many variables. The manager’s estimates of the information system’s accuracy would range from 100% to some lower level. Estimates that are made by the manager lead to a quantifiable dollar value associated with the information system to be developed.

Let us assume the manager foresees three product demand possibilities for the next planning period; a modest decrease in demand for the product, a modest increase, and a large increase in product demand. Changes in product demand combined with changes in production levels result in differing profit amounts. Profitabilities for specific actions are shown in Table 2. Table 3 is a contingency table showing the manager’s estimated accuracy of an information system to predict future product demand.

Possible Product Demands

Table 2

Expected Dollar Payoffs of Decision Making

Predicted Demand Level

Table 3

Probability of an Actual Demand Occurring Given a Predicted Demand

The accuracy of the information system is expressed in terms of the probability of an actual demand occurring given that a certain demand was forecast. Table 3 shows that the manager estimates a modest increase in demand will actually occur 15% of the time given a modest decrease in demand was predicted; probability(actual modest increase in demand | modest decrease predicted). Expressing these conditional probabilities in a format of "X occurs when Y was predicted" is an intuitive manner for managers to express information system accuracy since it relates to the sequence in which events take place.

The remaining information available to the manager is the probability of which product demand will actually occur (see Table 4). The probabilities might simply be the past history of demands or the manager may use his/her estimates. Once these values have been supplied there is sufficient information to calculate expected payoffs of decision making with and without the information system. The difference between the two values represents the economic value of using the information system.

Note that the economic value (the difference in expected payoffs) is the highest amount a manager would pay to develop the information system. Otherwise the information system would cost more than its benefit could generate.

History of Demand Level

Table 4

Probability of an Actual Demand Occurring

(i.e. marginal row values for Table 3)

The expected payoff of decision making without the information system is uncomplicated. For each possible action the manager may choose, there is a expected dollar payoff depending upon which product demand occurs. Summing the products of the payoff times the probability of a specific demand occurring yields the expected payoff for an action.

expected payoff (action_i) = S_{j=1 to J} [( $ payoff of action_i given demand_j) * P (demand_j)]

In the example above, the expected payoff of the ‘downsize’ action would be

($12,000 * .3) + ($5,000 * .6) + (-$3,000 * .1) = $6,300

The expected payoffs for this example’s actions without an information system would be

expected payoff (downsize action) = $6,300

expected payoff (single shift action) = $11,700

expected payoff (double shift action) = $5,100

expected payoff (plant addition action) = $8,000

Without an information system to guide the manager’s choice, the most profitable action would be to run a single shift and expect a $11,700 profit.

To calculate the payoffs using the information system requires an additional piece of information, i.e. the probabilities of predicting each demand level. Managers may intuitively feel the probability for predicting a demand level should be the same as the demand level actually occurring. However, the predicted probability equals the actual probability only when the information system is 100% accurate. Perfect information systems almost never exist but they provide an upper bound on the expected dollar benefits of an information system

To calculate the probabilities of each prediction, solve the conditional probabilities (Table 3) combined with the marginal probabilities of the rows (Table 4) as a set of simultaneous equations. In addition to the three rows in our example, a constraint must be added that ensures the probabilities of the predictions sum to one. A final constraint is that all probabilities are greater than or equal to 0. Under certain circumstances a manager’s estimated probabilities may present an infeasible set of equations indicating that predicted accuracy of the information system is inconsistent with established actual probabilities. In such a case the values in Table 3 would have to be changed.

S_{j=1 to J} [P(A_i|P_j) * P_j] = A_i and

S_{j=1 to J} P_j = 1 and

for _{j=1 to J} P_j > 0

where (a) P(A_i|P_j) represents the probability of an actual demand ‘i’

occurring given that demand ‘j’ was predicted and

(b) P_j represents the probability of demand ‘j’ being predicted

So the equations for our example are

.80 * P₁ + .05 * P₂ + .10 * P₃ = .3

.15 * P₁ + .85 * P₂ + .10 * P₃ = .6

.05 * P₁ + .10 * P₂ + .80 * P₃ = .1

P₁ + P₂ + P₃ = 1

P₁ > 0

P₂ > 0

P₃ > 0

A solution to the above simultaneous equations yields

P₁ = .332

P₂ = .645

P₃ = .024

At this point a decision tree can be produced as Schell [1986] outlines. Table 5 shows the decision tree for decision making without the information system and Table 6 shows the decision tree using an information system. The difference between the two expected payoffs is $16,463 - $11,700 = $4,763. That difference is the economic value of the information system.

The value of a perfect information system can be obtained from the information given since it is the value of a perfect, infallible information system minus the value of decision making without the information system. When the system is infallible the probability of predicting a specific demand is equal to the probability of that demand occurring. Multiplying these probabilities times the optimal action when the demand is known will produce the expected payoff of the perfect information system. The value of perfect information in our example is calculated as

(.3 * $12,000) + (.6 * $20,000) + (.1 * $35,000) = $19,100

which is $2,637 greater than the value of the imperfect system and $7,400 greater than the expected payoff without information.

Changes in expected dollar payoffs are calculated in the simulations of this research. In order to provide a consistent measure for statistical analysis, the observations of expected dollar payoffs for perfect and imperfect information systems were transformed into percentage gains over the expected dollar payoffs without information systems. The calculations for percentage increase in the example are calculated as

( $16,463 - $11,700 ) / $11,700 = 40.7% = percentage gain in value of imperfect information

and

( $19,100 - $11,700 ) / $11,700 = 63.2% = percentage gain in value of perfect information

Table 5

Decision Tree for Expected Payoff Without an Information System

Table 6

Decision Tree for Expected Payoff With an Information System

Determining the value of a specific information system is a necessary goal in itself. The ability to calculate the value allows us to make general statements about the value of information systems based upon simulation results. A series of four simulations were performed with each generating data concerning 10,000 information systems. The upper bound of accuracy for each simulation was 100% while the lower bounds were 60%, 70%, 80%, and 90% respectively. Probabilities for the accuracy and inaccuracy of predictions were randomly generated within these bounds. Ranges of information system accuracy were generated around the four levels in order to assess how gross levels of accuracy affected information system value.

Generated simulation data included all data in Tables 2 through 4. Table 5 and 6 values were calculated from the randomly generated data. For each information system the simulation generated

(1) a number of actions that the decision maker could take

(2) a number of states of nature (such as demand levels)

(3) dollar payoffs for each combination of action and state of nature

(4) probabilities of predictions, states of nature, and conditional probabilities

A uniform distribution generated the number of actions the manager could choose and the number of states that could occur; values from 2 to 5 were generated. Values in the expected dollar payoff table were anchored to a base value that was randomly generated. Each value in the payoff table was determined by multiplying the base value by a random number generator using a uniform distribution. Values were constrained to plus or minus three times the base value and one quarter of the values were randomly assigned negative values since decisions should be allowed to cause losses.

The minimum prediction accuracy of the information system was bounded for each simulation at 90%, 80%, 70% or 60%. This primary accuracy measure is defined as the average probability that a given state of nature will occur given that state of nature was predicted; i.e. the average value of main diagonal elements in the conditional probability table. The value of a perfect information system was calculated at the same time the imperfect information system value was calculated. Calculating the value of imperfect information systems addresses the "illusion of control" issue [Langer, 1975] that Davis and Kottemann [1994] find as a fault in what-if analyses. Explicitly stating both a perfect as well as imperfect information system value diminishes an information system user’s bias to overestimate his/her control of the situation.

In the example presented earlier, the average accuracy of the information system using main diagonal values from Table 3 is computed as

[ .80 + .85 + .80 ] / 3 = .817

A secondary measure of information system accuracy was calculated as the average absolute difference of related marginal probabilities. For example, the absolute difference between the marginal probability of predicting a moderate increase in demand and the marginal probability that a moderate increase in demand actually occurs. Predicted accuracy of specific states of nature (see the predicted probability values in Table 6) influence the second measure of accuracy. In our example the secondary measure of accuracy would be

( | .332 - .3 | + | .645 - .6 | + | .024 - .1 | ) / 3 = .00033

Two hypotheses about information systems value are addressed by the simulations. First, the accuracy of an information system is hypothesized to positively affect the economic value of the system. Second, when one action dominates all others (i.e. one decision action has the highest payoff regardless of which state is predicted) the value of the information system will diminish. The basis for these hypotheses is that information has value derived from the changes it causes in decision making. The hypotheses are formally stated as

H₁ : Expected payoffs from imperfect information systems will increase as the accuracy of these information systems increase

H₂: Expected payoffs from imperfect information systems are less for situations where one decision action dominates than for situations where no single action dominates.

Table 7 provides descriptive statistics of key information from the simulations. The figures relating to information system value are the percentage increase in expected dollar payoffs as calculated above. A striking feature is the decrease in the value of perfect as well as imperfect information systems when a single action dominates the expected payoff from decision making. Indeed, when a dominating action exists the expected payoff from an imperfect information system is zero for the simulations run.

Differences in expected payoff values of imperfect information systems are statistically significant at the .01 criteria between levels of minimum information system accuracy (i.e. 60%, 70%, 80%, and 90%). There is not a statistically significant difference between the values for expected payoffs of perfect information systems across the columns. This is expected since values of perfect information should not be affected by accuracy levels for imperfect information systems.

Expected payoffs of imperfect information systems increase by a statistically significant amount across the categories of systems accuracy. The expected payoffs of perfect information decrease as the accuracy of the information systems increase. This is counter-intuitive but it reflects that the potential for difference in expected payoff between perfect and imperfect information systems decreases as the accuracy of information systems increases. This evidence further supports H₁ that information system value increases with the accuracy of the information system.

The hypothesis that a dominating decision action will reduce the expected payoff of an information system is dramatically shown. Information systems add value to managers’ decisions when the decision is changed by the information system’s output. When an information system does not change a decision action then there is little room for increased expected value.

Table 7

Descriptive Statistics for Simulation Variables

The simulation results provide some explanation for criticism that information systems and information technology do not provide a net increase in productivity. Mean accuracies in the simulation categories ranged from about 76.5% to 94.9%. Note that the simulation showed no value for imperfect information systems when a dominating decision action occurred. Many managers rely upon their experience and anecdotal evidence; there will be enough experiences where the information system will not increase expected payoffs so that some managers will doubt their value.

Further distrust of the value of information systems comes from the belief that information systems are infallible. Across the categories of information system accuracy in the simulations the mean accuracy falls more than 18%. The accompanying expected payoffs drop from 36.5% to 22.5%. Information system projects are frequently touted on their potential benefits, the value of perfect information, but the actual payoff achieved from an imperfect information system may be a much lower. This fact, in conjunction with low benefits or no benefits when a dominating decision action occurs, add fuel to the argument that information system expenditures do not generate a net increase in worker productivity.

While the mean expected payoff of an information system is substantial, it is influenced by accuracy of the information system and by the possibility that a dominating action will surface. We must address the criticisms of information system value not only with overall statistics showing increased payoff but with methodologies for calculating expected payoffs for specific information systems.

Evidence of information system value has been reported at the macro, industry level. Such evidence is good and reflects academic rigor but it may not sway managers that deal with daily decisions concerning the development of specific information systems. Those managers require a straightforward methodology to quantitatively assess the potential economic benefits of a particular information system. The simulations and decision tree analyses provide ‘bottom-up’ evidence demonstrating the economic value of information systems on both the specific and the general levels.

More accurate information systems provide higher economic benefits than less accurate systems. Information systems with a mean accuracy of approximately 90% yield a economic benefit more than 30% greater than the expected payoff of decision making without information systems. Systems with a mean accuracy of approximately 75% yielded benefits of around 24%. These ‘bottom-up’ figures confirm the ‘top-down’ analyses of industry levels of spending and productivity.

The simulations also provide an explanation of why managers have doubts that information systems are economically beneficial. Less accurate information systems, such as those with a mean accuracy around 75%, result in a dominating decision action in about a quarter of the cases. When a dominant decision action exists, the value of the information system is greatly diminished.

• Bacon, C. J., "The Use of Decision Criteria in Selecting Information Systems/Technology Investments," MIS Quarterly (16:3), September 1992, pp. 335-354.
• Belcher, L. W. and Watson, H. J., "Assessing the Value of Conoco’s EIS," MIS Quarterly (17:3), September 1993, pp. 239-254.
• Brynjolfsson, Erik, "The Productivity Paradox of Information Technology," Communications of the ACM (36:12), December 1993, pp. 67-77.
• Brynjolfsson, Erik and Hitt, Lorin, "Paradox Lost? Firm-level Evidence on the Returns to Information Systems Spending," Management Science (42:4), April 1996, pp. 541-558.
• Chandler, J. S., "A Multiple Criteria Approach for Evaluating Information Systems," MIS Quarterly (6:1), March 1982, pp.61-74.
• Clemons, E. K., "Evaluation of Strategic Investments in Information Technology," Communications of the ACM (34:1), January 1991, pp. 22-36.
• Davis, Fred D. and Kottemann, Jeffrey E., "User Perceptions of Decision Support Effectiveness: Two Production Planning Experiments," Decision Sciences (25:1), January-February 1994, pp. 57-78.
• Due, Richard T., "The Value of Information," Information Systems Management (13:1), Winter 1996, p. 68.
• Hogue, J. T. and Watson, H. J., "Management’s Role in the Approval and Administration of Decision Support Systems," MIS Quarterly (7:2), June 1983, pp. 15-26.
• Keen, P. G. W., "Value Analysis: Justifying Decision Support Systems," MIS Quarterly (5:1), March 1981, pp. 1-15.
• Langer, E. J., "The Illusion of Control," Journal of Personality and Social Psychology (32), 1975, pp. 311-328.
• Leyland, F. P., Watson, R. T. and Kavan, C. B., "Service Quality: A Measure of Information Systems Effectiveness," MIS Quarterly (19:2), June 1995, pp. 173-188.
• Loveman, G., "Cash Drain, No Gain," Computerworld, November 25, 1991, pp. 69-72.
• Matlin, G., "What is the Value of Investment in Decision Support Systems?," MIS Quarterly (3:3), September 1979, pp. 5-34.
• Mukhopadhyay, T., Kekre, S. and Kalathur, S., "Business Value of Information Technology: A Study of Electronic Data Interchange," MIS Quarterly (19:2), June 1995, pp. 137-156.
• Panko, R. R., "Is Office Productivity Stagnant?," MIS Quarterly (15:2), June 1991, pp. 191-203.
• Roach, S. S., "Services Under Siege - The Restructuring Imperative," Harvard Business Review, September-October 1991, pp. 82-92.
• Schell, George P., "Establishing the Value of Information Systems," Interfaces (16:3), May-June 1986, pp. 82-89.
• Schumann, Matthias, "Methods of Quantifying the Value of Office Automation," Journal of Information Systems Management (6:4), Fall 1989, p.20.
• Smith, R. D., "Measuring the Intangible Benefits of Computer-Based Information Systems," Journal of Systems Management (34:9), September 1983, pp.22-27.
• van der Duyn Schouten, F. A., van Eijs, M. J. G. and Heuts, R. M. J., "The Value of Supplier Information to Improve Management of a Retailer’s Inventory," Decision Sciences (25:1), January-February 1994, pp. 1-14.
• Weill, P. and Olson, M. H., "Managing Investment in Information Technology: Mini Case Examples and Implications," MIS Quarterly (13:1), March 1989, pp. 3-17.

EJISE is published by Academic Conferences Limited
Curtis Farm, Kidmore End, Nr Reading RG4 9AY, England
Tel: +44 (0)1189 724148, Fax: +44 (0)1189 724691, Email: info@ejise.com

Send mail to info@academic-conferences.com with questions or comments about this web site.
Copyright © 2002-2006 Electronic Journal of Information Systems Evaluation
Last modified: November 25, 2004
ISSN 1566-6379