Background to the case study research:
Success and Failure of IT/IS implementation.

Research into IT/IS implementation in manufacturing indicates that most projects fail to deliver what is expected of them.  A large number of studies provide figures to indicate the levels of failure commonly experienced, and whilst reported levels have varied, the rates quoted are all worryingly high.  An early study by the American Production and Inventory Control Society (A.P.I.C.S.) in 1981 [1] found that over 76% of MRPII installations were perceived as unsuccessful by operations or production managers.  Other more recent reports quote similar high figures for failure rates.  Hochstrasser cited a general reported failure rate of up to 60%, in that within three years of implementation, the IT/IS was either discontinued or had not delivered the required benefits [2] .  Lyytinen and Hirschheim quoted a generally accepted failure rate of at least half information systems [3] .  Flowers argues that “total or partial failure of IS developments is endemic throughout the business world,” stating that even though such disasters happen frequently, surprisingly little may be known about the events that contribute to particular IS failures.  In trying to identify case studies of IT/IS failure, Flowers encountered an “industrial amnesia”, and referred to a “strange collusion” between the buyers and sellers of IT/IS [4] .  This perhaps contributing to the fact that many companies express surprise on hearing of the high levels of failure reported by research studies.  Clearly, the lack of openness prevents an open examination of the issues, and actively prevents organisational learning from former mistakes.

Failure Definition.

Consensus amongst researchers therefore, is that failure rates of IT/IS are high.  However, amongst the many studies and surveys in this field, there is no clear, accepted definition of failure, beyond subjective, personal interpretation.  Neither is there a clear idea within those manufacturing companies considering this issue, any clear, formal idea of what makes IT/IS successful.  In many cases, companies will judge success or failure according to “gut feel”.  Lyytinen and Hirschheim examined the concepts and complexities of failure, defining ultimate failure as the complete abandonment of all development, maintenance of the implemented IT/IS.  This absolute system failure or “termination failure” is obvious and well understood, since it is usually characterised either by a complete hardware crash, or the total inability of the software or operating system to cope with operational information flows.  However, beyond termination failure exist many differing degrees of IT/IS failure, and within individual companies there may exist some sort of informal consensus concerning the success or failure of its IT/IS.  This is usually based around some sort of generally accepted perception of the system and its application, and how it supports individual users and the business activities within the company.  Lyytinen and Hirschheim identified four major types of failure:

¨      correspondence failure – where the IT/IS does not match the specific planned objectives.
¨      process failure – where the IT/IS implementation process is not completed within planned time or cost.
¨      interaction failure – where the IT/IS is not used perhaps due to negative user attitudes.
¨      expectation failure – where the IT/IS does not match user expectations.

Arguably the first two types of failure are more easily identified since they should relate specifically to initial implementation plans or IT/IS specifications – assuming these exist formally.  However both interaction failure and expectation failure present greater problems since there is an implicit assumption that clear objectives can be set and measured, and that there is some common understanding and assessment of user expectations and user attitudes.  Sauer argues that expectation failure is a fundamentally flawed concept since some expectations of an IT/IS are more reasonable than others [5] .  In addition to the range of expectations that may exist – reasonable or otherwise – IT/IS implementation can increasingly involve a wider number of stakeholders and a larger user-base, as more and more systems develop to support inter-company activities, and customer and supplier links.  User expectations are consequently extremely varied amongst those who have a stake in the proposed IT/IS, and will always be hard evaluate and measure.

Organisational factors influencing success of IT/IS.

Early research concentrated mainly upon the failings of management, with Lucas  concluding that it was because social and behavioural factors had often been ignored that projects were often perceived as failing [6] .  Since then many researchers have built upon Lucas’ conclusions, providing a number of papers on user involvement and user perceptions of IT/IS.  Inevitably, organisational and human resource factors have been identified as being highly influential.  Symons and Raymond both emphasised the importance of user involvement and user perceptions as being critical to the perceived success of IT/IS [7] [8] .  Raymond in particular, examined the organisational characteristics associated with success.  Factors such as past experience of IT/IS, the proportion of internally developed applications, the degree of centralisation, top management support, user participation and intensity and rigour of the IT/IS requirements analysis were all recognised as being significant.

Further empirical studies have concentrated on the identification and examination of facilitators and inhibitors within organisations [9] [10] .  Internal and external factors identified as facilitating and influencing the success of IT/IS implementation, included strong technical support and expertise, the leadership position in IT, competitive pressure, strong financial position, extensive computerised facilities [11] .  However, factors that negatively influence the ability of an organisation to successfully use IT/IS – termed organisational inhibitors – have been studied rather less.  According to King, lack of appropriate planning, lack of top management support, difficulty in assessing tangible contributions are all major inhibitors.  Some studies have especially concentrated on the ability of SMEs to benefit from IT/IS.  SMEs are often recognised as being “poorer” than larger organisations in terms of human, financial and material resources to support successful IT/IS implementation [12] .  Fink reviewed research in SMEs and summarised those factors identified as facilitating IT/IS adoption in SMEs. Although many of the studies cited were inconsistent in terms of company size, commonly recognised factors included:

¨      Organisational size

¨      MD attitude to IT/IS

¨      MD knowledge of IT/IS

¨      External pressures to adopt

¨      Perceived benefits

¨      Competitive pressure

¨      Level of control and degree of planning for IT/IS implementation [13]

Evidence also suggests that increasingly SMEs are faced with more complex IT/IS investment decisions.  In some cases this is because of market trends in hardware and software, and also because major customers or parent companies “impose” a system, or  impose a requirement to integrate systems.  Ballantine reported that prime drivers when selecting and implementing IT/IS in SMEs were pressure from major customers, and an emphasis on improving efficiency [14]

Problems with evaluating and justifying proposed IT/IS investment.

The high failure rates cited above suggest that there is a wide gap between the level of investment in IT/IS, and company ability to achieve the necessary benefits from such investment.  Evaluation is defined as “establishing by quantitative and/or qualitative means the worth of IT to the organisation” [15] .  However, the evaluation or investment appraisal of IT/IS is problematic because of the difficulties associated with the identification and measurement of the benefits and costs associated with such investments.  As a result of this most companies do not formally evaluate their IT/IS investment.  Hochstrasser reported that only 16% of companies used “rigorous methods to evaluate and prioritised their IT/IS investment” and found that where investment appraisal of IT/IS did take place, it was usually based upon financial techniques specifically designed to assess financial impact in terms of cost [16] .  A survey by Peat Marwick McLintock in 1989 indicated that “44% of top UK companies and public sector organisations made no attempt to quantify the benefits of IT/IS investment,” despite the average spend on IT/IS being just under 2% of turnover [17] .  A more recent study by Ezingeard et al indicated that over half of the companies participating in the study did not formally list the benefits expected of the IT/IS investment, but “justified the investment as an ‘act of faith’ ” [18] .  Interestingly whilst most companies involved in this study were not satisfied with the evaluation techniques employed, they mostly offered views on whether they felt value for money was achieved with their system investment.

However a number of researchers have recognised that the range of tools available for organisations to use in practice is limited, and the traditional, accounting-oriented, cost-benefit analysis is far too narrow to adequately evaluate IT/IS [17] [2] [19] .  Although the response of users to the introduction of IT/IS is widely accepted as critical to its eventual success or failure, it is rarely taken into account as part of the accepted investment appraisal procedure.  Hinton and Kaye argued that wider organisational aspects should be used to understand the investment justification process more fully, and have observed that decision-makers investing in different business areas often adopted different perspectives [20] .   IT/IS investments, like operations, were usually characterised by a short-term perspective, and justified by an established financial appraisal technique such as Return on Investment.  In contrast to this, marketing investments were accompanied by a longer-term strategic perspective.  Decision-makers were able to incorporate a strategic dimension into their process of justification, including justification statements such as – improved market share, perception of company in the market place – and rarely, if ever, using financial justification techniques.

Whilst this contrast in investment justification perspectives does appear to exist, the use of financial evaluation methods for IT/IS does encourage companies to take a short-term perspective on investments.  Increasingly IT/IS projects within manufacturing environments are designed to improve the medium to long-term business objectives of a company.  Such business objectives need to be measured both quantitatively and qualitatively.  Benefits of a system designed to cut costs can be measured relatively easily in quantitative, financial terms.  However IT/IS projects which aim to improve customer support or offer better market information might be impossible to quantify short-term.  As more and more IT projects fall into the latter category, conflicts arise in assessing the value of investments and ongoing support costs, designed to support medium and long-term strategies, by using short-term financial techniques.  Hochstrasser asks how can we assess investments that are medium to long-term, risk intensive, and aimed predominantly at qualitative improvements?  Hence there is a clear need to evaluate and assess the true business value of IT/IS investments, and to monitor the performance of the investment over time.

Of five commonly recognised complaints about IT/IS in Europe and the US, two specifically relate to IT/IS investments:

(i) “IT/IS investments are unrelated to business strategy”,

(ii) “Payoff from IT/IS investments is inadequate”.

Clearly, the evaluation of IT/IS investments relies upon the satisfactory assessment of costs and benefits, and this assumes that the full costs and benefits of such investments are well understood.  Hinton and Kaye refer to the iceberg effect of “hidden costs” where the intangible qualitative costs are difficult to estimate accurately.  Amongst these are the ongoing support costs associated with the technology.  The Gartner group (see Hinton et.al.) estimated in 1995 that 73% of total costs of computer ownership were in support costs – clearly a figure that most investing in IT/IS would find extremely high.  This aspect of ongoing support costs is one that needs further investigation, since they form a substantial proportion of any IT/IS investment, but are often very difficult for most companies to evaluate prior to implementation.

Classification of IT/IS.

Since the evaluation of IT/IS is so complex and the range of systems and technologies now available are also increasingly complex, it is useful to classify IT/IS in order to identify different methods of investment justification according to IT/IS type.  Powell identified a classification of evaluation methods, which recognised objective and subjective evaluation methods (see Ballantine).  The objective approaches identifying values related to inputs and outputs of IT/IS, whilst the subjective methods considered the wider aspects to evaluation such as user attitudes. Hogbin and Thomas suggested a broad classification of four types of IT/IS projects according to the function:

¨      operational transaction processing systems including order processing
¨      systems which manage company resources across the organisation such as inventory and production systems
¨      systems which specifically target company growth and increased market share
¨      systems of strategic importance – which can often be a combination of all three above [21]

Hochstrasser also suggested a number of IT/IS project group types, each of which could  be justified and measured by a related set of evaluation criteria.  This classification is more detailed than that provided by Hogbin and Thomas and has the advantage of suggesting that specific evaluation criteria may vary according to IT/IS group type.  Examples include internal IT projects which are aimed at increased internal efficiency, and thus evaluated by improved work practice.  External IT/IS projects which tend to be measured by business performance indicators such as market share and perception of company image, and can be linked specifically to longer term business objectives [2] .  Investigation within the case study companies will attempt to examine whether IT/IS evaluation does differ according to the classification of IT/IS.

The case study research.

The case study based research now currently being undertaken aims to provide an in-depth analysis of a small number of SMEs in a number of topics related to IT/IS investment, and to explore the management process of this investment by those companies.  Initial work began with a series of semi-structured interview with key managers within the participating organisation.  Preliminary interviews gathered basic information on the size, structure and commercial activities of the company, as well as clarifying the types of IT/IS within the company and the background to these systems.  Follow-on interviews were held with managers working in different areas of the company, and explored managers’ perceptions of company IT/IS, what they considered company attitude to investment in IT/IS was, and individual and company attitude on whether company IT/IS was successful.  As much information and informal discussion was recording during the informal interviews in the following broad areas:

§         Existing systems

§         Proposed systems

§         Mechanics/processes for exploring/specifying/selecting/implementing IT/IS

§         Company attitude to IT/IS

§         Company organisational characteristics 

In addition to the wide range of data and information on IT/IS, an attempt was made to identify organisational characteristics according to the company strategy type.  The typology used was first defined by Miles and Snow, who suggested that organisations could be grouped into one of four types which illustrated how company management responded to the company’s competitive environment to determine company strategy.  The four types observed and identified by Miles and Snow are briefly described below:

§         Defenders – organisations which have narrow product market domains, and by concentrating on this narrow focus seek to improve the efficiency of their existing operations.

§         Prospectors – organisations with a strong interest in product or market innovation, who continually search for market opportunities.  Such companies are often identified as instigators change or uncertainty, and as such are not always deemed to be efficient operators.

§         Analysers – organisations which may be operating in a variety of environments, but who monitor, analyse and respond to new ideas.

§         Reactors – organisations in which managers observe and react to change and uncertainty, but do not always respond effectively to such change [22] .

This classification has been extensively discussed and used [23] [24] since it was first proposed, and in this study the development of the typology by Conant et. al. has been used.  This examines the company strategic typology from three groups of strategic problems and solutions – entrepreneurial, engineering and administrative (see Table One).  The questionnaire developed and tested  by Conant comprises eleven questions based around these three problems and solution sets, and has subsequently been used in other more recent studies, including the work undertaken by Henderson [25] .  The questionnaire provides the identification of “organisational strategy characteristics” in this case study work.  It establishes an important framework upon which data can be examined relating to company attitude to investment in IT/IS, type of IT/IS selected and implemented, and the process employed to justify and evaluate the IT/IS investment.

The company typology questionnaire was completed individually by all managers interviewed, in order to provide a management perspective of the company management strategy.  In addition, the researcher also provided an external viewpoint by completed the same questionnaire following the initial case work.  The preliminary findings presented here relate to one company only, and conclude by summarising particular areas which need further investigation, both within this organisation, and also in other case study companies to be studied.

Table One.

Preliminary findings at Company A.

Company A is a Small to Medium Sized company of one hundred employees.  The company is a “young” company, having been established as a joint venture between two companies – one UK-based, and the other Japanese.  Although the company is part of a larger group, it operates in an autonomous way.  The Operations Director is the key decision-maker on-site, reporting ultimately to the Board of directors at the parent company some two hundred miles away.  A number of managers report to the Operation Director, and functionally take responsibility for the Production, Commercial, IT/IS activities.  Annual turnover was £28 million last financial year, with a forecast increase to £30 million for this current financial year.

The background to the company IT/IS was historically dictated by the parent company in that some fifteen years ago, when Company A was established, the IT/IS was imported from the parent company, which operates in the same industrial sector.  The last four to

five years have seen a significant change in the company policy on IT/IS, as five years ago, the company sought to justify and establish internal expertise in IT/IS, and to take

total responsibility for future developments.  This appears to have been a policy which was proposed and developed by the current Operations Director, and for the last four years the company has operated with a small internal IT department.  In this time the company has continued to run the inherited database system which functionally controls internal functions such as finance (and still links with the parent company systems for this function), and basic stock holding.  Perceptions and comments on this system were that it was nearing the end of its life cycle, and that whilst the system could be “unwieldy”, it had been made to work reasonably well.  Additional systems in existence include a number of small applications developed for individual use, and based upon the networked PCs within the company.  These applications generally date from the establishment of the in-house IT expertise and were mostly originally aimed as short-term “quick-fix” solutions for particular end-user needs.

An additional system implemented one year ago, was a joint development with the company’s major customer.  This system was implemented to provide links between the companies, supporting the ordering and despatch process, and to support purchasing and production planning within Company A.  Current and future plans for other IT/IS developments are based upon a five year strategy plan set by the Operations Director, and  include ongoing investigation of a similar system to be linked to another strategically important customer, and current phased implementation of a shop floor data collection (SFDC) and warehousing system.  Comments during interviews with all personnel indicated that all managers had been involved with the ongoing specification and development of the SFDC system, and that all had a clear perception of the drive or need for such a system.  The strategic aims of the system were identified by most managers interviewed as:

§         the need to move to a paperless environment

§         the need to provide improved informational support for commercial activities.

It was observed that the desire to eliminate paperwork within company operations was a particular objective of the Operations Director, and that this strategic aim was well communicated to, and supported by, management and their departments.

The resulting general perception of IT/IS within the company is positive.  All departmental managers interviewed felt that their needs had been fairly well catered for, or at least that current and planned IT/IS developments would specifically provide them with useful applications, or improved information sources.  Responses of most interviewed indicated that they felt company attitude to IT/IS was positive and that IT requirements were often viewed positively and justified if a suitable case was made.  Most managers interviewed were not actively involved in the justification process.  Both the IT manager and the Operations Director had more direct involvement, with the IT manager taking responsibility for the costing of the specific hardware and software elements, and the Operations Manager undertaking the “formal” IT justification and gaining formal Board approval for any IT proposals.  The process of justification and approval can be summarised briefly as follows:

Once IT/IS specifications and alternatives have been investigated, formal costs are compiled for hardware and software element of the development.  The Operations Director then formulates a case to present to the Board of Directors.  This involves taking the initial given cost estimate, and doubling it in order to work through a financial justification exercise.  The Operations Director described this as:

“It’s just an experience thing.  I don’t know why double, but multiply by two and you can guarantee you won’t be far out.  Obviously if we then come in on budget, we get our saving quicker.”

However, as well as the financial justification based upon double the estimated costs, there was considerable evidence that other long-term business benefits were taken into account when evaluating the IT/IS both before and after implementation.  The investment justification process was described by the Operations Director as “rigorous,” and other benefits identified a key to the company were:

§         Increased tie-in with strategically important customers

§         Competitive edge gained by implementing innovative systems (newly gained business of 20% share in a new industry sector was quoted as a direct benefit of company IT/IS).

§         Increased business from a sister company of the customer involved in the joint IT/IS development implemented one year ago.

§         Long-term business improvement measured generally by financial contribution per employee, which has increased over the last year by 10-15%.

The company typology questionnaire was evaluated using the responses of the operations Director and the researcher.  The two response sets are illustrated in Table 2 below.  Using the decision ruling determined by Conant, the following strategy types are identified:

The predominant strategy type identified is that of Prospector, highlighting the company emphasis on the importance of marketing and being customer oriented.  The entrepreneurial questions indicated that company type was predominantly Analyser, indicating an awareness and evaluation of market trends, and adoption of new ideas where appropriate.  Overall typology differed between the two sets of responses for the engineering or technological business aspects – reactor and prospector, although some answers included responses in line with prospector and analyser behaviour.  Use of the other managers’ responses, and how much consensus existed amongst managers regarding company strategy type, will be discussed in a later paper.

Table Two.