Introduction

A legacy system is made up of technical components and social factors (such as software, people, skills, business processes) which no longer meets the needs of its business environment. The study of legacy systems has tended to be biased towards a software engineering perspective and to concentrate on technical properties. This paper suggests that the evaluation of potential change options for legacy systems can only be carried out as part of an holistic organisational analysis. That is, the evaluation of legacy systems must take place within a framework that combines business and technical considerations. We have, therefore, developed an inter-disciplinary approach that brings together an organisational scenarios tool (based on concepts from the field of organisational development) and a technical scenarios tool (based on concepts from the field of software engineering). These tools are applied in an iterative way, so that technical options are tested out against the business needs. It is, thus, a dynamic tool that seeks to mimic the nature of organisational change, as far as is practicable.

The empirical research took place within a large UK-based company that is structured around a call centre operation. They are referred to hereafter as CallCentre. The research team were asked to examine a particular job management function of the business. Major problems were being encountered with the flow of work through the business process chain. It was thought that this was primarily due to human error although CallCentre also made it clear that the technology involved was inadequate to support the complexity and size of the tasks being performed. The SABA approach (Brooke, 2000; Ramage et al, 2000) was applied to examine the main areas of concern and to identify potential for change.  An overview of the approach is given in Figure 1.

Figure 1. Overview of the SABA Approach

The Empirical Work

We held four meetings with CallCentre’s management, attended a one-day internal workshop and conducted a one-day workshop, both with a wide range of staff, followed up by a further half-day session with management. The SABA approach itself was applied during the one-day dedicated workshop and half-day follow-up.

The objective of this paper is to illustrate the output from the SABA workshop and to highlight some of the interesting questions and issues it raised. The material presented here is intended as a substantial introduction to the application of the approach. The case study raises some important points and indicates the usefulness of the approach.

The SABA approach stipulates that participants should include (but are not restricted to) technical and business experts in order to be able to generate the information required at each stage. Ideally, a participant group should consist of about a dozen people and include:

·      senior directors (preferably including someone at Board level)

·      managers from different organisational functions (including Human Resources)

·      IT specialists (preferably including a software engineer)

·      front-line staff (including those at the external customer interface)

·      end users (preferably including an external customer)

The one-day workshop at CallCentre attracted good attendance - 16 participants in all (in addition to the three research members). The only group we were unable to involve directly was the external customer although customer representatives were present.

Getting Started

Referring back to Figure 1, the first Organisational Scenarios Tool stage begins with helping participants to describe their organisation as it currently exists. An icebreaker exercise, such as asking participants to illustrate on paper their personal view of the legacy system, can be a useful way to begin. Once this is done, all the participants are asked to generate the first scenario, referred to as the status quo. Following this, participants are given three new scenarios to consider which are, in themselves, stereotypes but which encourage participants to be creative in thinking about organisational change. On completion of the status quo exercise, participants are divided up into small groups and facilitated to develop their own scenarios. Ideally each group should produce three or four additional scenarios. When this has been achieved, the facilitators bring the groups back together again to evaluate the status quo and the new scenarios. Participants are asked to prioritise the scenarios in order to avoid an explosion of possibilities before moving onto the next stage. It is important to note, though, that the status quo scenario must always be carried forward to the next stage of the model for comparison, as the objective is to assess the gap between what is currently in place and possibilities for the future, as well as to assess the feasibility of any suggested changes.

The Organisational Scenarios Tool

The Organisational Scenarios Tool was developed by Brooke (2000) and inspired by the research of Clegg et al (1996).

The day at CallCentre began with a description of the SABA approach and was followed by an icebreaker exercise. The exercise consisted of individuals expressing their personal view of the current set-up and its associated problems in the form of a picture or diagram. Discussion then took place in a plenary session of the different perspectives. Interestingly, there was a general consensus concerning the nature of the problems. These focused around issues such as fragmented communications, inaccurate information (their word), inadequate technology, and a general lack of quality control. After this the group was asked to generate a description of the status quo using the OST matrix (see Table 1). This exercise was facilitated by a member of the research team and recorded on paper how they viewed the current set-up. A brief description now follows of the criteria in the matrix (more detail of the theoretical underpinning is given in Brooke, 2000).

Boundary refers to the scope of the business area under evaluation. It could be a particular department, for example, or it could be the whole organisation. The boundary for the analysis at CallCentre was a particular job management activity and the chain of processes associated with it. The information systems supporting this activity ran in a virtuous circle from the customer through to professionals in the business and back out again to the customer. The participants viewed the processes as a chain through which (ideally) information flowed speedily and without error. They spoke a great deal about the fluidity of information and problems were seen in terms of the interruption of this flow-through and this perspective is echoed in the scenarios they produced.

Vision in this context does not refer to a mission statement or the like. It refers to a global summary of how the organisational unit being evaluated actually exists. Participants were asked to consider why it was that the job management activity had been designed in its particular way. What was the vision behind its design and function? Responses suggested that the design to a large extent had simply followed history in a “because we do it that way” sense. It was recognised that the core of the business was about delivering a service and it was thought that designing the processes in this way enabled the service to be delivered with a minimum of problems. The vision had been to remove problems away from the customer interface. It had become clear to the business though that the vision had lost its legitimacy; hence the need for a re-evaluation.

Logic refers to the rationale that underpins and drives this arrangement. The organisation centralised the control of its job management functions hoping, thereby, to achieve greater efficiency. Also, the intention was to “remove the dross” from the highly paid professional staff. The organisational structure flows from this. The role of the job management function was to support professionals in delivering a service to the customer. In order to do this more quickly, service delivery was done on a regional basis, hence the whole process chain was a mixture of centralised and regionalised controls. A problem highlighted at this point was that there was no dependency built into the system. Too much of the activity ran in parallel so that problems often did not become apparent until much later ‘downstream’ in the process chain.

Work roles were fairly mixed. Some roles in the process were de-skilled (e.g. one group of technicians were no longer able to manage their own individuals job schedules) whereas for other groups there was a certain amount of multi-skilling. Specialist skills were still required, mainly only for the higher paid professionals. Key to an understanding of the role of information systems vis-à-vis people at CallCentre was their view of information.

View of information asks the participants to identify the company’s general approach to the concept of information. The research team pointed to the difference between data and information and the difference between an objective view and one that gives more importance to the role of the individual in its interpretation. The responses from organisational members were self-contradictory. Whilst individual members recognised that ‘information’ (i.e. not data) received on a terminal screen would not be interpreted in the same way by everybody, they still insisted that this was a result of ‘human error’ and that the solution was to automate people out of the process as far as possible. The assumption was that the technology was a neutral vehicle for transference of facts and that it could speed processes across spatial and functional boundaries. The information systems role was to transmit what they called ‘facts’ unchanged from place to place and, therefore, with a high degree of accuracy. Information was seen as an objectified, externally manageable physical resource. This view of information is referred to shorthand in the SABA approach as ‘a resource view’. Its theoretical aspects are discussed in more detail elsewhere and it has links with the resource-based view of strategic management (Brooke, 2000).

The last three criteria addressed in the status quo matrix are costs, benefits and risks. These tend to be fairly familiar concepts and usually focus on economic considerations. However, participants were encouraged to think more broadly about effects associated with the existing set-up and to articulate both tangible (e.g. compensation pay-outs to customers) and intangible (e.g. damage to reputation) issues. The comments made in the matrix for these three criteria are fairly self-explanatory, except perhaps one: the cost of failure as a benefit. This refers to the effect that failure has on the increased demand for customer service. In other words, if the company delivers a service to a customer but does so incorrectly then it has another opportunity

to visit them to put it right. Since getting to the customer was seen as a current strength of the business (even if not performing the job entirely correctly) the company could maintain some

aspects of its good reputation. This ironic situation, however, is reflected badly under the costs section, where it was clear that re-work was a major issue. As one participant put it:

“We are good at fixing problems but we introduce a lot of problems, too. 50% of the work is actually re-work.”

 Table 1: Status Quo Scenario

Having assisted the participants to record their view of the status quo, the research team led them to briefly consider three alternative stereotypical scenarios labelled automate, informate, and transform. The purpose of these three stereotypes was to alert participants to the different ways in which technology could be applied to support, change or enhance business processes and, in particular, to pay attention to the different sets of values and assumptions which underpin the different strategies. The terms ‘automate’, ‘informate’ and ‘transform’ have become relatively well established in the literature, but a useful guide is given in Cash et al:

“When information technology substitutes for human effort, it automates a task or process. When information technology augments human effort, it informates a task or process. When information technology restructures,

it transforms a set of tasks or processes.” (Cash et al, 1994, see Preface, p.v.)

The concepts described therein encouraged participants to be creative in thinking about organisational change. Following this discussion, participants were divided into two smaller groups each facilitated by a member of the research team to develop their own alternatives to the status quo set-up. Each group was asked to generate at least three scenarios. Group One produced three alternative scenarios and Group Two produced four scenarios. For reasons of

space the details of these scenarios are not presented here. Each group was asked to rank their scenarios in order of priority. Returning to the plenary group, the participants presented to each other the details of the seven new scenarios and their reasons for their relative prioritisations. It became apparent that the preferred scenario from each group had some overlaps and were, in fact, fairly easily merged. The whole group felt that to produce this one final consensus view of change was the best way to move forward into the next stage (the Technical Scenarios Tool) of the approach. The scenario resulting from the end of this first OST stage is presented in Table 2.

 Table 2: The Prioritised Scenario

The Technical Scenarios Tool

Having looked at the way the organisation could change strategically in the future, the next phase of the approach looks at the possibilities for technology change for each of the scenarios. In particular, this requires analysis of existing IT systems which are regarded within the organisation as legacy. It is possible to approach this analysis in various ways, and a number of techniques have been used through the SABA project. The overall framework for the decision model is described as the Technical Scenarios Tool. This is illustrated in Figure 2, somewhat linearly - in practice the activities shown happen concurrently.

The TST stage in the CallCentre workshop was structured as a form of gap analysis. The analysis was conducted in a plenary group and was divided into two sections: problems (equivalent to information capture in Figure 2) and solutions (equivalent to solution routes).

 Figure 2: The Technical Scenarios Tool

 The first stage of information capture involves analysing in more detail the asset base that was discovered during the OST stage. This can include detail of the software itself (such as languages, structure, change history, documentation, data organisation) as well as details of the software maintenance processes, the staff undertaking these, domain information, and the role of the software in the organisation. In our experience, it is possible to home in quickly on the underlying problem (e.g. data usage), and to avoid wasting time on irrelevant matters.

We looked at the prioritised scenario and asked questions about what was preventing the

organisation actually getting from the status quo to the new scenario. It is worth noting here the importance of the status quo scenario generated at the outset. It should always be carried forward for comparison because the objective of the method is to assess the gap between what is and what could be, as well as to assess practicality of suggested changes.

The plenary group generated a list of problems associated with closing the gap between the status quo and the scenario in Table 2. We then went back over this list and tried to group the issues in order to tackle their solutions more easily. The groups which participants suggested were: technical problems, process issues, and organisational issues. For reasons of commercial confidentiality these details are not provided here but some general observations will be made to indicate the insights that the exercise provided.

During the information capture stage, six technical problems were identified, eleven process issues and five organisational issues. It became apparent that whereas process issues were by far the most significant to tackle, the technical solutions were less serious than had been expected. Indeed, it seemed that some of the anticipated problems would be eradicated once the process issues had been tackled. One of the most important insights from the exercise so far was that what appeared on the surface to CallCentre to be an information systems problem turned out to be much more to do with a need for change in work organisation and organisational attitude. Although at one level this appeared to challenge their expectations it also had positive outcomes as we will see.

Next we moved on to look at potential solution routes. By conducting the analysis in this way the participants came to see for themselves the social and organisational nature of the problems. It also had the benefit of making the changes seem more doable. There had been a concern at the outset that very expensive technological solutions would be necessary. Once the solutions routes had been identified it was clear that this was not the case. However, many of the changes needed depended on senior management intervention. Examples included adjustment of the scorecard methods of weighting performance criteria, and adjusting priorities for individual target setting. The participants had been under pressure in their various work roles to come up with improvements in business performance. Being able to present senior management with a detailed and holistic plan of actions that needed to be taken and which made senior management’s role explicit helped to relieve some of that pressure. It was felt that a good business case could now be made for implementing the changes. 

The later stages of the TST (analysis and details of solutions) were conducted at the company’s request with a group of managers who had responsibility for the areas in question.  This exercise was carried out during a half-day follow-up workshop and was combined with the second iteration of the SABA approach.   

Second Iteration of the SABA Approach

The overall SABA model (Figure 1) calls for another iteration of the OST stage once the TST has been worked through and a set of organisational and technical scenarios built. The goal of this second iteration is to evaluate the implementation details for their potential future business impact. Doing this through a second iteration of the model is a key feature of the SABA approach. A major objective is to help an organisation assess its future ability to change. This attempt at ‘future proofing’ makes a significant contribution to the evaluation of organisational change, particularly at strategic levels.

In our work with CallCentre, this occurred eight weeks after the one-day workshop described at the beginning of the paper.  The length of the gap was partly due to availability of all the participants, and partly to give the CallCentre managers time to consider the different scenarios and the technical problems & solutions in greater depth. Going into this meeting we had a set of organisational scenarios (from the OST), a prioritised scenario with its set of preferred solutions (technical, process and organisational) and a business case for its implementation from the TST stage.  

At the beginning of the meeting we revisited the prioritised scenario (Table 2).  It was felt by the managers that in order to strengthen the future-proofing exercise we should also return to the other seven scenarios generated by the small groups to make sure that we did not miss something.  When we did this, we found that three scenarios presented themselves as helpful in further developing the prioritised scenario.  The result of this further development is presented in Table 3.  The justification for developing the final scenario in this way is an essential part of the SABA approach.  In the second iteration of the OST the objective was to future proof the prioritised scenario by examining the market in which CallCentre operates and checking whether it would allow the company to meet the needs of that market. By looking at the combination of different change options, technical feasibility, and the wider organisational environment, we helped CallCentre to produce a final scenario which supports their need for on-going flexibility.

The final steps of the process are largely dependent upon the management at CallCentre.  In order to build a good business case for change, there needs to be a thorough consideration of the associated costs.  Once figures are available from CallCentre, the research team will assist them to put together a detailed implementation plan with timescales for presentation to the Board.  Success at this stage will involve ‘selling’ the final scenario to the Board and to others within the organisation.  This challenge will be made somewhat easier because of the participation of such a wide range of people in the workshops.

Conclusions

During our research on the SABA project we have found that organisations are biased towards physical, tangible and technically-oriented issues and that this is reflected in an inherent bias in established methods of information systems evaluation. It was, therefore, unsurprising that the mapping of the status quo revealed a tendency towards a resource view of information within CallCentre. The framework presented in this paper was developed with the objective of redressing this imbalance by exposing the values underpinning different choices and by centring

Table 3: Organisational Scenarios Tool Second Iteration – ‘Future Proofing’

on human factors in the development of technology. The fact that the participants identified for themselves the over-riding social and organisational nature of their problems helped to make the case more convincingly for them. However, it will still be largely dependent upon senior management to enable implementation of the changes. Also, full recognition of the human and social nature of their problems implies the need for a change in culture. This is a complex and sensitive area but the authors are of the opinion that without such recognition, many beneficial effects of organisational change will be lost. These issues were discussed with CallCentre during the final stages of the evaluation exercise.

 This research makes a contribution to information systems evaluation by presenting a framework that encourages organisations to consider different organisational scenarios and their underlying values and assumptions so that decisions can be made in a more explicit and informed way. One advantage of the framework is that it is intended to be a tool-in-use. The precise steps of the framework can be (and has been) tailored to the needs of the organisation. For example, the costs, benefits and risks criteria can be modified to reflect the organisation’s own project profiling methods, and criteria can be weighted accordingly.

From the start, it has been our aim to adopt an action-research perspective. Working with CallCentre has led us to reflect on the way in which we have designed our approach to helping organisations look at their legacy systems.  One thing that emerged very clearly was that the SABA model is definitely a tool-in-use, to be modified according to the needs of the situation.  It cannot be picked up ‘off the shelf’ and used mechanistically within an organisation.  In working with CallCentre we made several modifications to the SABA approach.  We tried to be responsive to the time made available to us and the level of knowledge of the participants.  The choice of categories for ordering the problems and solutions in the TST stage (organisational, process and technical) arose from the participants and the need to structure a set of rather disparate issues. We would be wary, however, of imposing this framework on every application of the TST, as its strength was the way it arose naturally from the participants.  Another change we made was to cut down the amount of time we spent working through the three stereotyped scenarios (automate, informate, transform) at the beginning of the OST stage.  While this certainly met CallCentre’s request to spend more time on scenario generation, with hindsight the lack of emphasis on the stereotypes was not ideal.  Insufficient emphasis of the stereotypes can lead to greater conservatism in the final scenarios produced and although this does not appear to have been the case at CallCentre, in reality we will never know the extent of its impact.   Such issues serve to remind us of the reality of conducting empirical research within a dynamic commercial context. 

Perhaps most importantly, the outputs from the SABA exercise form a valuable archive of an organisation’s decision-making activities.  Opportunities for more in-depth and critical post-implementation review have now been enabled.  The research team hopes to continue to support the evaluation of technological and organisational change at CallCentre into the future.

Acknowledgements

The other academic member of the research team involved in the CallCentre research work was Professor Keith Bennett, Research Centre for Software Evolution at the University of Durham. The SABA project was funded for three years between 1997-2000 as part of the SEBPC Programme by the Engineering and Physical Sciences Research Council.

Special thanks are due to the staff and management at CallCentre who invited us into their organisation and contributed much of their time, skills and knowledge to the research.

References

• Brooke C (2000) A framework for evaluating organisational choice and process redesign issues. Journal of Information Technology 15(1), 17-28.

•  

• Cash JI, Eccles RG, Nohria N and Nolan RL (1994) Building the Information-Age Organisation: structure, control and information technologies. Richard D Irwin, Boston.

•  

• Clegg C, Coleman P, Hornby P, Maclaren R, Robson J, Carey N and Symon G (1996) Tools to incorporate some psychological and organisational issues during the development of computer-based systems. Ergonomics 39(3), 482-511.

• Ramage M, Brooke C, Bennett K and Munro M (2000) Combining organisational and technical change in finding solutions to legacy systems. In Systems Engineering for Business Process Change (Henderson P, Ed), pp 79-90, Springer, London.

EJISE is published by Academic Conferences Limited
Curtis Farm, Kidmore End, Nr Reading RG4 9AY, England
Tel: +44 (0)1189 724148, Fax: +44 (0)1189 724691, Email: info@ejise.com

Send mail to info@academic-conferences.com with questions or comments about this web site.
Copyright © 2002-2006 Electronic Journal of Information Systems Evaluation
Last modified: November 25, 2004
ISSN 1566-6379