|
1. Introduction
In recent years
organisations have witnessed significant increases in the scale,
complexity, connectivity and strategic focus of information technology.
The current trend is for information systems, which are highly
integrated, have the ability to disseminate information throughout, and
between, organisations, and often have become an integral part of the
actual product or service provided [Doherty and King, 1998]. Such trends
mean that the introduction of information systems within organisations
almost invariably results in a wide variety of impacts upon the design
of the business, its economic performance and the working conditions of
members of staff; technical change is the catalyst for organisational
change. For example, the introduction of an enterprise resource planning
system (ERP), within a manufacturing company, is likely to have a
significant impact on that organisation's business processes, structure,
culture and enterprise level performance, as well as the motivation, job
specifications and performance of individual employees. It is suggested
that whilst many impacts can be classified as planned outcomes, others
are incidental side effects that may be of a positive or negative nature
[Doherty & King, 2002].
Unfortunately,
information systems development projects have been typically been viewed
as exercises in technical change, rather than socio-technical change,
and consequently a system's organisational impacts are rarely explicitly
reviewed and proactively managed [Clegg, 2000]. As Eason [1988; p 44]
has noted, 'traditional approaches to the development of information
systems have concentrated on the delivery of the technology, rather than
emphasising the human and organisational changes that are required in
order to ensure that the system delivers meaningful benefits'. This is a
dangerous strategy, because unforeseen and unresolved negative impacts
may increase the likelihood of systems failure, whilst potential
beneficial impacts may not be fully realised without an appropriate
programme of organisational change. Indeed, there is a growing body of
evidence to suggest that the high levels of systems failure witnessed in
recent years, estimates range between 30%-70%, can be largely blamed on
the failure of organisations to address the business change and the
wider organizational issues [Doherty & King, 2001].
One increasingly
important mechanism for ensuring that planned impacts come to fruition
is the benefits realisation management programme. This can be defined
as: 'the process of organising and managing, such that the potential
benefits arising from the use of IT are actually realised' [Ward &
Elvin, 1999]. Typically such programmes focus upon the organisational
change that is necessary to facilitate benefits realisation, rather than
the system's functionality. As Ward et al [1996] note, it is becoming
increasingly recognised that benefits are generally derived from the
organisational change that accompanies the introduction of IT, rather
than from the functionality provided by the IT. Whilst a benefits
realisation programme is typically focused upon planned impacts, it is
likely that by maintaining a focus on benefits throughout a project many
incidental impacts should also be identified and proactively managed.
Whilst there is a growing recognition that IT projects should focus upon
the realisation of business benefits, rather than the delivery of a
technical solution, there has, to date, been relatively little research
to explore how a benefits realisation programme can best be undertaken.
One potentially interesting line of enquiry is to establish the
characteristics of the organisational capability required to succeed in
realising benefits from IT investments. A research project, seeking to
address this issue, was initiated, which has been broken down into the
following three major phases:
1
Framework Development: The objective of the initial phase of the study
was to develop a research framework, to guide the study as a whole. The
purpose of the framework, which was primarily based upon a thorough
review of the literature, was to present a high level, 'idealised'
vision of how benefits realisation should be applied in practice.
2
Exploratory Analysis: The aim of the second phase was to conduct an
exploratory analysis of the application of benefits realisation
management within a small sample of recently completed IS development
projects. Having collected and analysed the data, it was envisaged that
the 'best practice' framework would be used as a mechanism for exploring
the extent to which the reality of benefits realisation match 'best
practice'. More specifically, it would be possible to gain a deeper
understanding of:
a The
nature and impact of benefits realisation, in cases where its practice
did broadly match the framework.
b The
inhibiting factors and implications, in cases where the practice of
benefits management diverged significantly from the framework.
3
Case Studies: The aim of the third and
final phase of the research is to undertake some in-depth case studies
of the treatment of benefits realization in ‘live’ information systems
development projects. It is anticipated that for each of these cases,
data will be collected using a combination of observation, document
reviews and interviews. Moreover, it is envisaged that a representative
sample will be chosen to ensure that it includes organisations that have
explicit benefits realization programmes as well as those who have no
specific focus on benefits. The aim of this phase of the project is to
gain new insights into the capabilities necessary to support benefits
realization by comparing and contrasting the experiences of those
organizations that are successful in realizing benefits with those that
have been distinctly less successful.
It was envisaged
that the exploratory analysis [phase 2] would provide some important
insights to help understand the validity of the framework [phase 1], as
well as providing a sharper focus and an important backdrop for
conducting the full study [phase 3]. To date, the literature review has
been completed, a research model has been developed and the first,
exploratory phase of the empirical work has been commenced. The aim of
this paper is to present a progress update on this study by presenting
the literature and research model, before providing an exploratory
analysis of how benefits realization has been approached in a sample of
IS projects. The following section of this paper presents the research
framework, an analysis of the results of the initial exploratory study
is presented and discussed in the fourth section, before their
importance is assessed in the final section.
2. The research
framework
The research
framework has been derived primarily from an extensive review of the
literature. However, the experiences of one of the two researchers, who
is a very experienced IT developer and project manager, greatly helped
to shape its final form. To this end the researcher has kept a research
notebook in which he recorded observations and the results of document
reviews with respect to the application of benefits realisation
management on the projects in which he has been involved. The aim of
this section is to firstly describe the framework, before
contextualising it in the literature.
2.1 The
‘Best Practice’ framework for benefits realisation
Figure 1 provides
an overview of the research model. In outline the major elements of the
framework are as follows:
Technology
Delivery / Benefits Realisation Projects
The main unit of
analysis within the framework is the individual IT project, each of
which is conceived as having a technology delivery and a benefits
realisation component. As benefits realisation is the major focus of the
research we have chosen to sub-divide it into a number of distinct
phases, namely benefits planning, delivery and review. As there has been
little explicit discussion of these three concepts in the literature, we
start the following discussion of each by presenting a working
definition.
1.
Benefits Planning: We define benefits planning as 'the process of
identifying and enumerating the planned outcomes of an IS development
project and explicitly stipulating the means by which they will be
achieved'. As such, this process should operate at two distinct levels.
Firstly, the IS / IT Strategy should present a broad overview of how the
planned portfolio of IS applications will support the realisation of
business benefits, and in so doing directly contribute to corporate
objectives. Secondly, benefits planning should be conducted, in far more
detail, for every individual project. Murray et al [2001] note that the
typical flow from strategy to IT investment portfolio and then to
projects and subsequently change programmes is wrong. A major change is
required so that business change programmes are identified directly from
the strategy.
2.
Benefits Delivery: Benefits delivery can be defined as 'the execution of
the set of actions necessary to realise all of the benefits specified in
the benefits plan'. Consequently the process of benefits delivery
typically runs from project initiation, after approval of the business
case or benefit realisation plan, through to completion of the project.
Benefits delivery focuses upon the organisational change necessary to
facilitate benefits realisation, rather than the delivery of the
technical solution. As Strassman [1990; p 519] notes 'computers add
value only if surrounded by appropriate policy, strategy, methods for
monitoring results, talented and committed people, sound relationships
and well designed information systems'.
3.
Benefits Review: Benefits review can be defined as the 'process by
which: the success of the project in terms of benefit delivery is
assessed; opportunities for the realisation of further benefits are
identified; and lessons learned and opportunities for improvement in
future projects are identified'. Benefits review is therefore conceived
as being an ideal opportunity for organisational learning, so that the
organisation's capability to succeed in the realisation of benefits can
over a period of time be enhanced.
Figure 1:
Conceptual Model for Research
Organisational
Context
Projects take
place in a dynamic organisational context. There are two broad areas to
consider. Firstly, as the goal of a project is to improve organisational
performance and provide benefits to organisational stakeholders it needs
to be considered in the context of the organisational strategy, which
sets the priorities for change and improvement. Secondly, as projects
typically result in significant changes to the organisation, it is
important to understand the relevant characteristics of the organisation
being changed.
Benefits
Realisation Capability
In the past
twenty years there has been significant interest in whether
organisations can assemble a unique portfolio of resources, competencies
and capabilities that will render them a competitive advantage. For
example, the ‘resource-based theory of the firm’ [Wernefelt, 1984;
Barney, 1991] suggests that organisations will invest in those assets
and resources that they believe will best assist them in being
successful and gaining a sustainable competitive advantage. Similarly,
Prahalad & Hamel [1990] argue that an organisation will only attain a
competitive advantage if it can develop a set of core competencies that
competitors will finds it difficult to imitate. Whilst the literature is
not consistent in its use of many of the terms involved, it is possible
to discern the following broad distinctions:
§
Resources: The term resource is used in a very inclusive way – it covers
not just all the usual elements such as money, people, skills, capital
etc but also ‘softer’ resources such as credibility, brand, IP, and
procedures.
§
Competences: A competence is an attribute that is held at the
organisational level, as opposed to a skill, which is typically thought
to be held at the individual level. An organisation is competent if it
manages and deploys its resources effectively and efficiently.
§
Capabilities: These are typically viewed as the combination of
competences and resources deployed in the market place such that
customers see the organisation as having some competitive advantage.
Whilst the
interest in resources, competencies and capabilities was originally
restricted to the corporate level, more recently, the idea that there is
a broad set IS competencies, which give an organisation the 'capability'
to deliver effective information systems, is gaining much support in the
literature. For example, studies have attempted to establish the nature
of the IS capability. In two articles, Feeny and Willcocks (1998a;
1998b) explore the concept of capabilities and propose a framework of
nine distinct capabilities necessary for an effective IS function. More
recently, an empirical study by Santhanaman and Hartono, (2003) has
indicated a strong relationship between an organisation’s IS / IT
capabilities, its overall performance and its ability to secure a
sustained advantage. However, Santhanaman and Hartono, (2003)
acknowledge the simplicity of their measurement of IS capabilities and
flag-up the need for improved measures to be developed for use in future
studies.
It is important
to make the following points, to clarify how the term capabilities is
being used within the proposed research framework. Firstly, in our
research framework, we have deliberately chosen the term capabilities,
to reflect that there might be a mix of competencies and resources that
combine to facilitate the effective management of benefits. Secondly, it
should be noted that whilst the capabilities exist at an organizational
level, they are applied at the project level.
Benefits
Realisation Learning
There is a
growing acceptance that organisational learning should be an important
element of IS / IT management [Ward & Peppard, 2002; p 135]. It is
likely that the ability of organisations to learn will greatly
contribute to their success at realising benefits from IT investments,
but this issue needs to be far more explicitly explored through
empirical research. In formulating the research framework we have tried
to represent the interdependent nature of the relationship between
capabilities and learning. Whilst the work of Hamel & Prahalad (1990;
1993) has already emphasised that organisational learning depends upon
an organisation’s capabilities, our diagram highlights the likelihood
that organisational learning about the benefits realisation process
might also encourage an organisation to ultimately modify its
capabilities.
2.2
Contextualising the research framework
The review of the
existing literature has played an important role in shaping the
framework. In particular the 'framework for managing IT-enabled business
change' developed by Ward & Elvin [1999] has been particularly
influential. Aside from this contribution, the literature review has
revealed that there has been relatively little explicit coverage of the
content, process and impact of benefits realisation management, and
there is, therefore, a pressing need for further studies that adopt a
clear benefits’ perspective. It is envisaged that the proposed study
will also make an important contribution from a capabilities and
competencies perspective. As discussed in the previous section, there
are significant literatures already in existence that focus upon either
corporate competencies / capabilities, or the sub-set of this literature
that specifically targets the capabilities of the IS / IT function [see
figure 2]. However, the focus of our research forms an important, yet
largely neglected, area on the boundary between these two existing
bodies of literature, namely the capabilities necessary to facilitate
benefits realisation [shown as the shaded area in figure 2]. The only
previous study, to be identified, that explicitly tackles benefits
realization, in the context of competencies, concluded that
organisation’s need to develop competencies in ‘exploitation planning’
and ‘benefit evaluation’ if they are to realise value from there IS
investments (Peppard et al, 2000). Whilst the work of Peppard et al
[2001] has played an important role in helping to shape and position our
study, its scope was far broader than ours as it investigated a
comprehensive range of ‘information competencies’. By contrast we are
aiming to focus solely upon benefits realization capabilities and
subject these to far more detailed scrutiny.
Figure 2: Focus
of Research
Whilst the
justification for this study is at least in part, the relative paucity
of literature explicitly addressing the topic of benefits realisation
there are many contributions, from related bodies of literature,
particularly IT evaluation, that are directly related to our study. For
example, the following contemporary views of the nature and role of IT
evaluation suggest that it has much in common with benefits realisation:
§
Evaluation should be explicitly concerned with the on-going management
and direction of the project, and in so doing the realisation of
information systems' value [Palvia et al, 2001; Remenyi &
Sherwood-Smith, 1999]. As Farbey et al [1993] note, with respect to IT
evaluation, 'benefits have to be managed if they are to be gathered''.
§
Evaluation should also be concerned with assessing the process of
systems development, as well its product. Palvia et al [2001] and
Remenyi & Sherwood-Smith [1999] argue that the experiences of systems
development process should be explicitly evaluated, so that they can be
improved over time.
§ Whilst
the traditional view has conceived evaluation to be an activity that it
is undertaken in two distinct 'acts' [Remenyi et al, 1997; p 72], there
is a growing consensus that evaluation should be performed as an
on-going process. As Remenyi and Sherwood-Smith [1999; p 18] note: 'to
obtain most benefit from information system evaluation it is important
to conduct evaluations regularly as an integral part of the systems
development life-cycle'.
The
socio-technical literature, which is concerned with the human and
organisational aspects of systems development projects, also addresses
many of the same issues as the benefits realisation literature. For
example, a recent definition of organisational issues: ‘those issues
which need to be treated during the systems development process to
ensure that individual human, wider social and economic impacts of the
resultant computer-based information systems are likely to be desirable’
[Doherty & King, 2001], highlights the degree of overlap. The process of
explicitly addressing organisational impacts is likely to help ensure
that benefits are proactively managed and ultimately realised.
Consequently, whilst the literature on benefits realisation might be
fairly modest at present, there are plenty of contributions from the
evaluation and socio-technical literature that have strongly influenced
our thinking with respect to the research framework. Indeed, it is
possible to suggest that the IT evaluation, social-technical approaches
and benefits realisation literatures should be viewed as alternative
windows onto a similar set of concerns.
3. The initial
exploratory study
The aim of this
section of the paper is to describe the research approach applied during
this phase, before summarizing the results of the exploratory study. It
should be noted, that as we are currently extending the size of the
sample, this paper represents a progress report, rather than the
finished product.
3.1
Research approach
Access was gained
to the knowledge management system of a large software company, with
extensive IT consultancy experience. The knowledge-base contains
detailed records of the vast majority of the projects in which the
organisation's consulting arm played a significant role in the
development and implementation of IT solutions on behalf of, or in
collaboration with, commercial clients. Ultimately, a sample of sixteen
project histories was chosen for inclusion in the initial exploratory
analysis described in this paper. The projects were chosen as they were
the most highly ranked projects in the knowledge base, on basis of value
/ reusability, on the day of selection (22 / April / 2003). The rankings
are provided by the consultants (users of the repository) and the
subject matter experts who manage content in key areas of the
repository. The chosen projects provide a broad coverage across sectors
and around the world from major organisations. It is also worth noting
that none of the chosen projects were fully outsourced developments, as
each involved working in joint team with client staff.
A set of
questions was developed based on the research framework. The questions
were conceived from the standpoint that an IT project is an exercise in
socio-technical change with the objective of delivering benefits to the
organisation and organisational stakeholders. The sample being studied
comprises individual projects so the focus of the review is at the
project level rather than programme or portfolio perspectives. The
questions were focussed on three distinct aspects of benefits
realisation management, namely: benefits planning; benefits delivery and
benefits review. For each of the 16 projects selected the full set of
documents stored in the knowledge base was downloaded. For each project
the documents were reviewed in two stages:
§ An
initial review of all the documents was carried out to highlight areas
where the ‘best practice model’ had, or had not, been followed.
§ A
second review of the documentation was then carried out with the purpose
of responding to the specific questions that had been developed.
Following the
project by project review findings were summarised in a number of
different ways. The analysis of the findings presented in section 4
concentrates on an analysis of the common themes relating to each
specific question. In essence, this approach can be characterised as a
variable oriented, cross-case analysis [Miles & Huberman, 1994].
3.2
Research findings
The aim of this
section of the paper is to present a very summarized review of the
results of the exploratory, empirical analysis. This analysis focuses
primarily on the three component parts of benefits realisation, namely
planning, delivery, and review and this section is therefore structured
accordingly.
3.2.1
Benefits planning
The analysis of
the responses to individual questions addressed, with respect to
benefits planning, indicates that the projects were virtually all
focused on technology delivery rather than organisational change and
benefits realisation. Moreover, in no case were specific measures for
benefits defined, nor was there a clear linkage between benefits and the
project plan, nor any traceable link from the key features of the
technological solution back to the project's business objectives. It is
of course possible that the documentation reviewed had only addressed
the technical part of the projects, and any benefits’ focus in the wider
business project had therefore been excluded from the knowledge base.
However, if this had been the case, it is likely that would have been
some evidence of the benefits’ focus; for example, in setting the
priorities for the technical solution. On a more positive note, in a
number of cases there was some evidence of the need for organisational
change to accompany the implementation of a system. For example, in a
minority of the cases there was reference to business process or
workflow modifications, but these organisational changes weren’t
explicitly articulated on the project plan. It was also interesting to
note, that in all cases, the consulting company's standard development
approach was used to guide the project. Whilst in hindsight, it can be
seen that the use of this approach was generally appropriate, in some
cases it might have been tailored to better suit the organisational
context.
3.2.2
Benefits Delivery
The review of
development approaches, suggest that the companies have been adopting a
very typical project management approach, which emphasises the
aggressive management of time-scales, budgets and specified
functionality. However, whilst all of the reviewed projects were
successful, in terms of technology delivery, there was little explicit
focus upon benefits delivery and organisational change. To a very great
extent, the absence of a clear focus on benefits delivery can be
attributed to the failure to explicitly make anyone responsible for this
critical activity. In some cases the documentation did explicitly
identify a business owner or sponsor for the project, but did not make
any comment on the nature of their roles and responsibilities. On a more
positive note, there was often a good emphasis on the need for training,
but this was typically focused solely on how to use the system, rather
than on how to improve an individual's job performance.
3.2.3
Benefits review
Given the lack of
specific focus on benefits earlier in the project lifecycle the lack of
benefits related activities at this stage was not surprising. The
results of the analysis suggest that the projects were all successful as
technology projects. However, there is insufficient evidence to assess
their success in benefits terms. Indeed, it might be very difficult to
ever establish the success of these systems, in terms of benefits
delivery, as there is no evidence that the benefits of these projects
were ever explicitly appraised. In a number of cases, particularly for
the infrastructure projects, the outcome of the project was a new
capability / service that could be exploited in many ways. In one or two
cases, the project documentation highlighted the need for future
operational / management roles to own the services, but there was no
specific benefits / exploitation plan. It is also interesting to note
that as the project teams were typically disbanded as soon as the
solution had been implemented, there was no on-going provision for
evaluation or benefits review.
4. Concluding remarks
The broad aims of
this paper were to present a model of ‘best practice’ with respect to
benefits realization management, before presenting the provisional
results of an exploratory analysis of the practice of benefits
realization management, in a sample of recently competed IS development
projects. The initial empirical results have shown that there is a very
significant gap between the best practice framework and the practice of
benefits realization management. The majority of organizations and
projects studied adopted the traditional measures of project success,
namely delivery on time and on budget, and there was little evidence of
any explicit focus on benefits delivery or business change. However, the
innovative practices found in a limited number of organizations fitted
well with some key aspects of the proposed framework. The significant
gap found between current practice and the proposed benefits realisation
framework limited the feedback gained on the framework, but did serve to
highlight the importance of further work in this area. It can be
concluded that significant changes to the philosophy and practice of
systems development will be required to deliver a stronger focus on
benefits realisation.
This research is
important from a practitioner perspective as it underpins the need for
IT professionals to take a more proactive interest in benefits planning,
delivery and review. The study also has important implications for the
researcher, as it highlights the worrying gap between the theory and
practice of benefits realisation. The results of the literature review
are also important as they highlight the high degree of commonality
between the IT evaluation, socio-technical approaches and benefits
realisation management literatures. Consequently, it is important for
researchers to consult all three literatures when investigating the
organizational impacts of, and change requirements for, information
system’s projects.
The conduct of
exploratory study has identified a number of significant weaknesses in
our research approach. In particular, it has been difficult to draw many
inferences, from the knowledge base of IS projects, about the
organisational capabilities, the organisational learning, or indeed the
degree to which individual projects were successful in terms of benefits
realization. We are, therefore, currently in the process of arranging
interviews with project managers from each of the 16 projects to
investigate these important areas in more detail. Moreover, we are
planning to extend the scope of the initial exploratory study by
significantly increasing the sample size. On completion of the initial
exploratory study, it is envisaged that the ‘best practice’ framework
will be refined before commencing the in-depth case studies.
References
-
C. W. Clegg,
(2000) Socio-technical principles for system design. Applied
Ergonomics, 31, pp. 463 - 477.
-
Doherty, N. F.
& King, M., (1998) “The consideration of organizational issues during
the systems development process: an empirical analysis”, Behaviour &
Information Technology, 17 (1), pp 41-51.
-
Doherty, N. F.
& King, M., (2001) “An investigation of the factors affecting the
successful treatment of organizational issues in systems development
projects”. European Journal of Information Systems, 10, pp 147-160.
-
Doherty, N.F.
and King, M. (2002) ''From Technical Change to Socio-Technical Change:
Towards a Proactive Approach to the Treatment of Organisational
Issues'', in Socio-Technical and Human Cognition Elements of
Information Systems , Clarke, S., Coakes, E., Hunter, M.G. and Wenn,
A. (eds), Information Science Publishing, Hershley, PA, pp 22-40.
-
Eason, K.,
(1988), Information Technology and Organisational Change, Taylor &
Francis, London.
-
Ewusi-Mensah,
K. & Przasnyski, Z. (1994) “Factors Contributing to the Abandonment of
Information Systems Development Projects”, Journal of Information
Technology, 9, pp 185-201.
-
Farbey, B.,
Land, F. & Target, D. (1993) How to assess your IT investment,
Butterworth-Heinemann, Oxford.
-
Feeney, D. and
Willcocks, L. (1998a) “Core Capabilities for Exploiting Information
Technology”, Sloan Management Review, Spring, 39 (3)
-
Feeney, D. and
Willcocks, L. (1998b) “Re-designing the IS function around core
capabilities”, Long Range Planning, 31 (3).
-
Hamel, G. &
Prahalad, C. K. (1993) “Strategy as Stretch and Leverage”, Harvard
Business Review, 71 (2), pp 75-84.
-
Miles, M. B.,
and Huberman, A. M. (1994) Qualitative Data Analysis, Sage
Publications: Thousand Oaks, California.
-
Murray P et al
(2001), Advanced Benefits Management, Cranfield ISRC research report,
2001
-
Peppard, J.,
Lambert, R. and Edwards, C. (2000) “Whose job is it anyway?:
organizational information competencies for value creation”,
Information Systems Journal, 10 (4), pp. 291-322.
-
Palvia, S.,
Sharma, R. & Conrath, D. (2001) "A socio-technical framework for the
quality assessment of computer information systems", Industrial
Management & Data Systems, 101 (5), pp 237-251.
-
Prahalad, C. K.
& Hamel, G. (1990) “The Core Competencies of the Corporation”, Harvard
Business Review, 68 (3), pp 79-91.
-
Reymeni, D.,
Sherwood-Smith, M. & White, T. (1997) Achieving Maximum Value from
Information Systems: A Process Approach, John Wiley & Sons:
Chichester.
-
Reymeni, D. &
Sherwood-Smith (1999) "Maximise Information Systems Value by
Continuous Participative Evaluation", Logistics Information
Management, 12 (1/2), pp 145-156.
-
Santhanaman and
Hartono, (2003) “Issues Linking Information Technology Performance to
Firm Capability”, MIS Quarterly, 27 (1),
-
Strassman, P.,
(1990), The Business Value of Computers, The Information Economics
Press, Connecticut.
-
Wernerfelt, B.
(1984), 'A Resourced-based View of the Firm', Strategic Management
Journal, Vol 5: pp171-180
-
Ward, J.,
Taylor, P. & Bond, P. (1996) "Evaluation and the realisation of IS/IT
Benefits", European Journal of Information Systems, 4, pp 214-225.
-
Ward, J. &
Peppard, J. (2002) Strategic Planning for Information Systems, John
Wiley & Sons: Chichester.
-
Ward, J. &
Elvin, R. (1999) "A new framework for managing IT-enabled business
change", Information systems journal, 9 (3), pp 197-222.
|
