1.
Introduction
IT projects have been notorious
for their proneness to fail for some time (see, for example, Flowers
1996). In the United Kingdom, recently reported problems include delays
to an online hospital booking application in September 2004 (Arnott
2004a), to the national firearms database in October 2004 (Nash 2004)
and to the implementation of a secure national radio system for
ambulance and fire services in November 2004 (Arnott 2004b). A very well
known supermarket chain reported a write-off of £260 millions associated
with IT and supply chain systems (Knights 2004). The United Kingdom is
not alone in suffering from these setbacks. One survey of over 13,000 IT
Projects (Standish Group 2003) estimated that US corporations spent more
than $255 billion per year on software development projects of which $55
billion was wasted on failed projects. The project success rate was just
34%, while the project failure rate was 15%, with 51% of projects
suffering from cost overruns, time overruns, or a reduction in the
features and functions delivered.
One reason that has been put
forward for the prevalence of these failures is that information
systems/information technology (IS/IT) applications are not constrained
by physical laws (Brooks 1995). “Software is largely free of constraints
and its potential is therefore unlimited”, according to the Royal
Academy of Engineering (2004). Thus it is easy to embark on
over-ambitious and ill-advised projects. This is perhaps exemplified by
the supermarket project mentioned earlier: in that report (Knights 2004)
an analyst is quoted as suggesting that the organization “was too
ambitious on the business side, trying to over-segment its customer base
which meant that it, in turn, created an overly complex IT system that
wouldn’t scale.”
This paper proposes a method of
presenting in a visual fashion the factors that have a bearing on
project failure and their interrelationships. This allows the different
stakeholders in a project to use the diagram to collaborate in the
creation of risk models that can simulate the propagation and evolution
of risks throughout the project life cycle.
2.
Risk management
One textbook definition of risk
management is “the identification of the hazards and possible problems,
the evaluation of their importance and the drawing up of plans to
monitor and deal with those problems” (Hughes & Cotterell 2002, p.134).
Thus good risk management should be able to assist the reduction of the
likelihood of project failures. There are numerous risk management
paradigms, for example: Boehm 1991, Charette 1997, Dorofee et al 1996,
McManus 2004, Robin et al 2002 and Yardley 2002.
The elements of risk management
are summarised by Boehm (1991) as follows:
§
Risk Assessment.
·
Risk identification
·
Risk analysis
·
Risk prioritisation
§
Risk Control.
·
Risk-management
planning
·
Risk resolution
·
Risk monitoring
Risk management paradigms tend to
have a similar content, and most place an emphasis on adding details of
identified risks and their resolution to the lessons learned from
previously implemented projects. The maintenance of historical records
of risks and issues as they are closed allows an organisation to
effectively learn from experience (Boehm 1991, Dorofee et al 1996,
Charette 1997).
The definition of risks needs to
be conducted with care. There are conditions that might cause loss, such
as the inexperience of staff. Methods such as Riskit (Kontio et al 1998)
identify these possible causes of risk as risk factors. However, a risk
factor by itself does not identity a risk. There must, for example, be
many projects where some staff are inexperienced in some way but where
the project outcomes are successful. It also follows that the risk
factor of inexperience could lead to several different unfortunate
outcomes. For example, it could be that productivity is reduced in some
cases, and in other cases the quality of the end products is reduced. On
the other hand, a risk outcome like ‘delayed project completion’ by
itself is incomplete because in this case a risk outcome has been
identified, but not the risk factors. Thus Riskit also identifies risk
events which, when triggered in an environment where certain risk
factors are relevant, will cause certain risk outcomes.
3.
Learning in the risk management process
It has been suggested above that
an effective organizational learning cycle can increase the capability
and maturity levels of the team, project and organisation. However,
although risk management is focussed on identifying future problems, it
is usually difficult for people to foresee future events and problems (Wiegers
1998). The study of past projects, however, can help to ‘sensitise’
project participants to the potential obstacles to a new project’s
success. Top management should therefore support and sponsor the
evaluation of implemented projects in a ‘no-blame’ culture. After the
implementation stage it is time to learn from the collective experience
of the project team and to retain that knowledge, (McConnell 1997).
Pitagorsky (2000) sums this up neatly: “The most important step to
improve the quality of decision making is the Post-Implementation
Review.
The processes by which an
organisation can learn from past experiences are core elements of the
concept of the ‘learning organization’ (Argyris 1999). As Argyris points
out in his survey of the literature on learning organizations, some
writers have identified obstacles that prevent organizations using past
lessons as a basis for improving future performance. Leavitt and March
(1988) for example point out that organizations often adopt strategies
that have worked in the past but which do not work in new situations.
The lessons may be based on a small number of cases that might not in
fact be typical. The links between cause and effect in past projects may
not in fact be obvious or can be subject to controversy. What exactly
happened in a past project may not be clear, and judgements about the
relative success or failure of a project may depend on the viewpoint of
individual stakeholders.
These obstacles to effective
organizational learning do not in themselves invalidate the argument
that organizations need to learn from past experience. In fact it is
argued that they underline the need to make the nature of the lessons
learnt and the basis upon planning is based more explicit.
4.
Project risk evaluation and documentation
Post-Project Reviews,
Post-Mortems or Project Post-Evaluations all have one ultimate purpose:
to learn from past projects, be they successful, challenged or failed.
However, the terms used in such retrospective processes must be chosen
with care. For example some people have a particular perception of the
term ‘Post-Mortems’ where Spafford (2003) argues that “a failed project
is one thing, but to have a post-mortem meeting after a great project
just doesn’t sound good”. Even with acceptable shared perceptions that
are intended to emphasise the positive aspects of learning, the actual
processes involved need to be carefully designed. Dalcher (2003) states
that:
“Information about each failure and the circumstances
surrounding the failure are difficult to obtain, but there is also a
general lack of knowledge about the ways, methods and approaches for
doing so”
(Dalcher 2003).
This author goes further in
calling for new ways of “studying the failure phenomenon” supported by
empirical validation (Dalcher 2003). Ikram (2000) observes that risk
management itself has not benefited from rigorous research. In fact, the
author is quite critical about the claims made: “The current literature
provides useful knowledge and guidelines on Risk Management, but many of
the claims made in the literature have no empirical validation.
According to the empirical findings, the application of Risk Management
to Information Systems Development is not a common practice. “(Ikram
2000).
DeMarco and Lister (2003) suggest
that by applying post mortems to half a dozen past projects we could
have enough data as input for future risk management process.
5.
Longitudinal case study in IS project
failure
The proposed framework was
developed as the result of a longitudinal study of a problematic system
development and implementation project. An advantage of the case study
approach was that it added richness to the detailed information
collected. It was able to capture causal influences and interaction
effects which would not have been detected by a more statistical
approach (Garson 2004).
The longitudinal case studies (LCS)
method requires that quantitative/qualitative data are collected a
number of times from the area of study (Jensen and Rodgers 2001). In
this instance, the case study, which was embedded in a government
organization in Kuwait, was documented during several field
trips/visits. The project started in 1998-1999 and raised many failure
issues at the beginning of 2000. The project suffered from various
setbacks during the following two years. At one point the project was
stopped for a period of time, and many stakeholders thought that the
project had failed and been abandoned. The project was reinitiated and
went through much revision of the project design and management
approaches. Many problems and issues remain with the project up to the
current time.
5.1
First phase
The aim of the first phase of the
study was to document the a) background for the project and, b) the
shortfalls that led to project failure (Al-Shehab et al 2004). Data
gathering was done through semi-structured interviewing (Kane & Brun
2001) of the stakeholders using a taxonomy-based questionnaire (Dorofee
et al 1996).The analysis of the data collected in this way used a
grounded theory approach (Dick 2003). This was used to identify
qualitatively the concept variables relating to the relative success or
failure of the project mentioned in the interviews. These concept
variables were thus candidate risk factors.
The list of concept variables
identified by the stakeholders is shown below
§
High level
description
§
Unclear project
scope
§
Insufficient budget
estimation
§
Poor performance
contractor
§
Unclear evaluation
criteria
§
Contractor has no
previous experience in developing such projects
§
Unrealistic
schedule estimation
§
Delays in project.
§
Contract milestones
were not clear towards payment
§
Poor partnership
relation between customer and contractor.
§
Undefined user role
§
No user involvement
in the project
§
All-in-one project
§
Technical staff not
appropriately skilled to tackle technical activities required
§
No tech-team member
involvement
§
Wrong design
§
Undefined project
objective
§
Lack of project
management skills
§
Lack of project
control
§
New technology
introduced
§
Poor database
structure design.
§
Lack of leadership
§
lack of
communication
§
Lack of control
over contractor
§
Poor product
outcome
§
Unfrozen
requirement
§
Poor design
§
Poor documentation
§
Unstructured design
§
Project plan was
not followed
§
Lack of top
management support
§
Lack of project
guide and objective
§
Lack of user
commitment
§
Delays in
acceptance testing.
What is noticeable with this list
is that some are clearly risk outcomes, that is, the effects or
consequences of preceding problems, as in the case of ‘delays in
project’. It is also noticeable that the precise meaning of terms and
expressions used by various stakeholders may not be obvious, and that
problems associated with the lack of a shared and explicit ontology may
be quite important. Other identified variables do not themselves
necessarily point to the probability of project failure, but might
contribute to failure in conjunction with other factors, for example,
‘new technology introduced’. These would be identified as risk factors.
What is required is the identification of cause and effect relationships
between the risk factors and risk outcomes. Below is a list of the
relationships between risk factors and risk outcomes that were
identified by the stakeholders.
§
High level design
led to unclear project scope
§
Insufficient budget
estimation led to use of unqualified contractor.
§
Unclear evaluation
criteria led to not making sure the contractor has previous experience
in developing such projects.
§
Unrealistic
schedule estimation led to delays in project.
§
Contract milestones
were not clear towards payment, which led to poor partnership relation
between customer and contractor.
§
Undefined user role
led to no involvement in the project
§
An all-in-one
application design led to large number of technical tasks needing to be
completed compared with a small number of unskilled technical staff.
§
No user involvement
led to wrong design
§
Undefined project
objectives led to wrong design
§
Lack of project
management skills led to lack of project control
§
New technology
introduced led to poor structure design.
§
Lack of leadership
led to lack of communication
§
Lack of control
over contractor led to poor product outcome
§
Unfrozen
requirement led to unstructured design
§
Poor documentation
led to unstructured design
§
Project plan not
being followed led to lack of control over project.
§
Lack of top
management support led to unclear project scope
§
Lack of user
commitment led to delays in acceptance testing.
In this list it can be seen that
one risk outcome could be linked to several risk factors. For example,
‘wrong design’ is caused by a combination of ‘no team member
involvement’ and ‘undefined project objectives’. In other cases, the
outcome from one risk becomes a factor that contributes to some other.
For instance, ‘undefined user role led to no involvement in the project’
which in turn means ‘lack of user commitment led to delays in acceptance
testing’. The relationships between the factors that lead to
unsatisfactory project outcomes therefore seem to be more complex than
what the simple list of cause and effect relationships above implies. In
the next section the possibility of using causal maps to create a richer
picture of these relations will be explored.
5.2
Second phase
Experience during data collection
and subsequent analysis suggested the need for a method that could
capture the often complex interactions between concept variables in a
project environment. Causal cognitive maps (CCM) suggested themselves as
such a method. The second phase of the case study therefore focussed on
adopting CCM as a tool for, a) documenting the past experience of the
project, and b) forecasting the outcome of the project.
CCMs were drawn individually by
project team members in assisted sessions. These were then combined to
produce a consolidated map that was presented to and further analysed in
a group session. The participants were asked to provide their views on
the map and whether they agreed or disagreed with any of the
sub/individual maps. A large amount of data was collected, which is
currently being analysed as part of a subsequent investigation into the
application of quantitative model building. Currently a commercial tool,
Decision Explorer by Banxia, is being assessed for the use in this case
study.
5.3
Experimentation with and evaluation of relevant techniques
A particular aspiration of this
work is to reach agreement on a common ontology of project risk factors
and outcomes that could be applied to future projects. The project team
members who are involved in the study can be clustered into a management
group and a staff group, and the differences between the two need to be
assessed. An intriguing research question is whether CCMs produced by
the two groups will be different but complementary, that is focussing on
different factors but not actually contradicting each other. An
alternative is that there is actual contradiction where one group for
example perceives a positive influence between two factors when the
other does not.
Elements of this part of the
study are:
§
Refinement of
project maps in terms of the cause and effect chains that lead to
undesirable project outcomes.
§
Production of a
consolidated map by the management group in a collaborative session.
§
Production of a
consolidated staff map.
§
A comparison of the
resulting management and staff maps.
§
An investigation of
limitations of the tool and techniques.
6.
Causal and Cognitive Maps (CCM)
A causal map is a network diagram
representing causes and effects (Bryson et al 2004). The diagram
contains two basic elements: concepts, which are the nodes in the
network and causal relationships, represented by the arcs between the
nodes. Concepts are considered as the variables of the system and in
some notations carry either a positive or negative sign implying the
type of the causal relationship and effect (Tsadiras 1997). Cognitive
maps use the concept to elicit and represent perceptions.
CCMs have been used frequently in
the operations management discipline (Axelord 1976, Brown 1992, Bryson
et al 2004, Eden 1988, Scavarda et al 2004, Williams 1995),, commonly
using them to support empirical research for building and communicating
theory. The areas where the causal maps have been used include:
§
Risk mitigation:
anticipating unintended consequences.
§
Diagnosis:
identifying the possible causes of a problem.
Huff (1990) suggests that an
advantage of causal maps is that they can portray information about a
system more succinctly than a corresponding textual description.
Using CCM as the risk
identification approach appears to have some advantages:
1.
Group discussions
guided by CCMs encourage the participation of all relevant stakeholders
in the project.
2.
CCM-facilitated
group discussions tend enhance communication between the project members
(Gotterbarn, 2001).
3.
Using CCM provides
a clear picture of the project situation by creating a diagrammatic
representation.
4.
The diagram enables
identification of the interrelations between risks.
Figure 1:
A causal map for the case study in Section 5.1
7.
System Development Life Cycle (SDLC)
In conventional project
management, a project has to be broken down into physical activities to
which resources can be allocated. It is argued by the authors that every
IS/IT project follows some kind of life cycle that places structure on
the temporal order of the project’s activities. Admittedly in some
projects this order is not very evident, but there is a generic
development cycle that can be applied to most of IS/IT projects. This
cycle typically contains the following phases:
§
Initiation: system
concept, management approval, funding approval.
§
Planning/Procurement: identifying stakeholder, top-level project plan,
feasibility study, high-level view of the intended system and the
determination of its goals.
§
Design/Implementation: requirement definition, analysis, design
completion & approval, coding/unit test, integration/test,
system/acceptance test.
§
Installation/Deployment: installation complete, evaluation/acceptance,
site preparation, training, business process re-engineering, rollout.
§
Completed: review &
maintenance.
These stages are reflected in the
causal map of Figure 1.
The concept variables relating to
risk that have been identified in earlier sections are indicators of
some general conditions that could affect more than one activity and
these activities could be in more than one project phase. A lack of
project management skills could, for example, affect many different
activities at many different stages of the project.
However, despite the ubiquity of
some of the risk variables, others are specific to particular phases.
For example, in the case study, the risk ‘lack of user commitment led to
delays in acceptance testing’ clearly relates to one specific stage in a
project. A typical approach to risk management is to maintain a risk
register during a project. As the project progresses, some risks will
cease to have an impact, while the threat of others may grow. It has
been found helpful to produce a template separated into the generic
stages of the development life cycle and to locate the occurrence of
risk factors and their outcomes within these physical stages, as well as
linking the risk factors through chains of cause and effect.
8.
The way forward
It is widely accepted that
efforts to identify and avoid potential risks in IS projects are
commonly rewarded. Weigers (1998) states
“anything you can do to improve
your ability to avoid or minimize previous problems on future projects
will improve your company’s business success and reduce the chaos and
frustration that reduces the quality of worklife in so many software
organizations.”
This author goes on to make
specific suggestions about what needs to be done, and indeed how to go
about it. His suggestions are not limited to the documentation of risk
assessment, but also the actual perceptions of those involved in the
assessment process. Both informal and formal assessments are important,
as are the strategies employed in risk mitigation. Apart from the
obvious need to evaluate, identify and mitigate, the overall aim is for
an organisation to self-improve via the new knowledge gained, i.e. to
maintain and update the corporate knowledge base. The Department of
Defence (DoD) risk Management Guide for DoD Acquisition (2003) includes
a step called “Analogy Comparison/ Lessons-Learned Studies”. This step
uses lessons learned plus historical data to identify similar risks or
risk areas in the current project. They argue that any new project is
originated or evolved from existing projects or represent a new
combination of existing components or subsystems.
The availability of a high
quality historical knowledge base is therefore important. Causal maps
can offer far more information than purely textual documentation formats
and in a compact and highly-visual format (Huff 1990). Repositories of
historical project data are common, but large collections of detailed
information gathered during a project’s lifecycle are difficult and
time-consuming to analyse. There are many ways of gathering data
relating to past projects. Some employ surveys, interviews, emails or
reports to summarise the history of the project (McConnell 1997),
(Collier 1996) and (Wiegers 1998). However, the need for a more
structured method of documenting past projects in a more clear
representation, is vital.
It is important to recognise that
ontological issues are important in situations characterised by
disparate groups of people, with different roles and with potentially
different ways of expressing concepts. As Gruber (1993) points out, “One
problem is how to accommodate the stylistic and organizational
differences among representations while preserving declarative content.”
Stylistic differences relate in our case studies to the language
structures used. In our opinion, a common ontology for risk is required,
and the issues involved should be conveyed to potential users of causal
mapping techniques. The measurement of risk factors and their
interdependences also offers exciting possibilities for elicitation,
modelling and simulation.
9.
Further work
9.1
Risk modelling techniques
The use of CCMs in the management
of IS/IT project risk is currently not widespread. In this context, the
authors see the use of CCMs as a means of making visible the perceptions
of project stakeholders with regard to the causes of shortcomings in
completed IS/IT development projects. While no map can be justifiably
claimed to be completely accurate in its identification of the source of
unwanted project outcomes, they can promote debate and further
investigation. This in turn should improve managerial decision-making.
Two major challenges are clear.
The first is to move from the diagnosis of the sources of past problems
to the prediction and forecasting of potential problems in new projects.
While stakeholders may be able to identify such factors as ‘insufficient
budget’ or ‘inappropriate experience’ as contributors to project
shortfalls in completed projects, the identification of these risk
factors in the proposals for new projects is not necessarily
straightforward.
9.2
Measurement of risk’s cause and effect
The risk assessment process,
guided by the process of causal mapping, introduces the concept of
expressing the degree to which a risk factor exists in a project, and
also the impact such a risk factor has (or is likely to have) on related
risks. One approach of measuring the impact is by using a coloration
approach (Aladwani 2002). In the authors’ experience, stakeholders are
commonly unable or unwilling to give precise values to these, but are
usually more open to expressing them as an (expert) opinion, i.e. as a
‘fuzzy’ value (Negnevitsky 2002) that does not commit the participant to
likely inaccuracy. In spite of potential inaccuracies, in the authors’
experience, expert judgment, tacit though it may be, is valuable and
quite often close to reality. Fuzzy representation and reasoning
approaches (Kosko 1986, Tsadiras and Margaritis 1997) and neuro-fuzzy
and system dynamics (Rodrigues 2001) techniques may well hold the key to
usefully capturing such risk data, and making the ensuing models
functional.
10.
Conclusions
The framework described in this
paper can be used to document the knowledge that team members gained
through experience. This knowledge can be presented in a single map,
essentially visualising the ‘big picture’ of the past project as a
pedagogical post-evaluation method. The elicitation process involves the
identification of risk factors, their likelihood of occurrence, and
their likely impact on other risks within the cause and effect
relationships at the heart of the model. The use of CCMs throughout this
process has been observed to facilitate ‘brainstorming’ and to achieve
consensus in a more intuitive and effective way than with more
conventional approaches. The availability of an agreed general risk
model of a specific project is also useful, not only in evaluating the
accuracy of past diagnoses, but also in providing the means to develop
and challenge stakeholders’ mental models of perceived future events.
11.
Acknowledgment
We would like to thank those of
our colleagues who have contributed to this work, especially Tim Brady
and Nick Marshal of CENTRIM at University of Brighton.
References
-
Aladwani, A
(2002). “IT project uncertainty, planning and success: An empirical
investigation from Kuwait”, Information Technology & People, Vol. 15
No. 3, 2002, pp. 210-226.
-
Al-Shehab,
A, Hughes, B, and Winstanley, G (2004) “Using Causal Mapping Methods
to Identify and Analyse Risk in Information System Projects as a
Post-Evaluation Process”, Proceedings of ECITE 2004, Amsterdam.
-
Argyris, C
(1999) On organizational learning. (2nd Edition) Blackwell, London
-
Arnott, S.
(2004a) “Delays hit NHS bookings trial” Computing 2 September p1
-
Arnott, S.
(2004b) “Emergency services face network delays” Computing 18 November
p1
-
Boehm, B
(1991) “Software Risk Management: Principles and Practices”, IEEE
Software, pp 32-41.
-
Bryson, M,
Ackermann, F and Finn, C (2004), Visible Thinking: Unlocking causal
mapping for practical business results, John Wiley & Sons, Ltd. 2004.
-
Brooks, F
(1995) “No Silver Bullet-Essence and Accident” In The Mythical Man
Month and Other Essays on Software Engineering, Anniversary Edition,
Addison Wesley.
-
Carr, M,
Konda, S, Monarch, I, Ulrich, F C and Walker, C F (1993)
“Taxonomy-Based Risk Identification”, [online] Carnegie Mellon
University, Software Engineering Institute. CMU/SEI-93-TR-6,
ESC-TR-93-183. <http://www.sei.cmu.edu/pub/documents/93.reports/pdf/tr06.93.pdf>
-
Charette, R
(1997) “Managing the Risks in Information Systems and Technology”,
Advances in Computers, University of Maryland. Volume (44) page 1-58.
-
Collier, B,
Demarco, T and Fearey, P (1996). “A Defined Process For Project
Postmortem Review”, IEEE Software, pages 65-71.
-
Dalcher, D
and Drevin, L (2003). “Learning from information systems failures by
using narrative and ante-narrative methods”, Proceedings of the 2003
annual research conference of the South African Institute of Computer
Scientists and Information Technologists on Enablement through
technology.
-
DeMarco, T
and Lister, T (2003) Waltzing with Bears: Managing Risk Software on
Software Projects Dorset House Publishing, New York, USA.
-
Department
of Defense, Defense Acquisition University (2003) “Risk Management
Guide for DoD Acquisition”, [online], http://www.dau.mil/pubs/gdbks/risk_management.asp
-
Dick, B
(2003) “Grounded theory: a thumbnail sketch”, [online], http://www.scu.edu.au/schools/gcm/ar/arp/grounded.html
-
Dorofee, A,
Walker, J, Alberts, C, Higuera, R, Murphy, R and Williams, R. (1996)
Continuous Risk Management Guidebook. Software Engineering Institute,
Carnegie Mellon University, USA.
-
Flowers, S
(1996) Software Failure: Management Failure, John Wiley & Sons.
-
Garson, D
(2004), PA 765 Statnotes: An Online Textbook,
http://www2.chass.ncsu.edu/garson/pa765/statnote.htm
-
Gotterbarn,
D (2001) “Enhancing Risk Analysis Using Software Development Impact
Statements”, Proceedings of the 26th Annual NASA Goddard Software
Engineering Workshop, November 27-29, Greenbelt Maryland
-
Gruber, T
(1993) “A Translational Approach to Portable Ontology
Specifications.”, Knowledge Systems Laboratory technical Report KSL
92-71, Stanford University, CA.
-
Huff, A
(1990), Mapping Strategic Thought., Wiley and Sons, USA
-
Hughes, B
and Cotterell, M. (2002). Software Project Management. 3rd edition.
McGraw-Hill Companies, UK.
-
Ikram, N
(2000). “The Management of Risk in Information Systems Development”,
PhD desertion, University of Salford
-
Jensen, J L
and Rodgers, R (2001). Cumulating the intellectual gold of case study
research. Public Administration Review 61(2): 236-246.
-
Kane, E and
Brun, O’Reilly-de (2001) Doing Your Own Research, Marion Boyars
Publishers, United States and Great Britain.
-
Knights, M
(2004) “Sainsbury’s dumps inefficient systems” Computing 21st October
p1
-
Kontio, J
Getto, G. and Landes, D. (1998). “Experiences in improving risk
management processes using the concepts of the Riskit method”. SIGSOFT
’98 Sixth International Symposium on the Foundations of Software
Engineering
-
Kosko, B
(1986) “Fuzzy cognitive maps”, International Journal of Man-Machine
Studies. 24, 65-75.
-
Leavitt, B,
and March, J G (1988) “Organzational learning”, Annual Review of
Sociology 14
-
Mcmanus, J
(2004) Risk Management in Software Development Projects, Elsevier
Butterworth Heinemann
-
McConnell, S
(1997) Software Project Survival Guide. Microsoft Press, Washington
-
Nash, E
(2004) “Firearms database delayed once again” Computing 28th October
p1
-
Negnevitsky,
M (2002) Artificial Intelligence a Guide to Intelligent Systems.
Addison Wiesley, England.
-
Pitagorsky,
G (2000) PM Network, Project Management Institute, March, pp 35-39.
-
Robin, A,
Preedy, D, Campbel, D, Paschino, E, Hargrave, L, Born, M, Huber, N,
Haynes, P, Kazmi, P Oikawa, R and Getchell, S (2002). “Microsoft
Solutions Framework - MSF Risk Management Discipline v.1.1”, [online]
<http://www.microsoft.com/technet/itsolutions/techguide/msf/msrmd11.mspx>
-
Rodrigues, A
(2001) “Managing and Modelling Project Risk Dynamics A System
Dynamics-based Framework”, PMI Europe 2001, London UK, 6-7 June, pp1.
-
Scavarda, A,
Bouzdin-Chameeva, T, Goldstein, S, Hays, J and Hill, A (2004) “A
Review of the Causal Mapping Practice and Research Literature”, Second
World Conference on POM and 15th Annual POM Conference, Cancun,
Mexico.
-
Standish
Group International, Inc. (2003) “CHAOS Chronicles report”, Yarmouth,
Massachusetts, USA, [online] < http://www.standishgroup.com/press/article.php?id=2>
-
Smith, J
(2001) “Troubled IT projects: prevention and turnaround”, Institution
of Electrical Engineers, UK
-
Tsadiras, A
amd Margaritis, K (1997). “Cognitive Mapping and Certainty Neuron
Cognitive Maps”, Information Sciences 101, pp109-130.
-
Wiegers, K
(1998). Know Your Enemy: Software Risk Management. Software
Development magazine, October 1998.
-
Yardley, D
(2002) Successful IT Project Delivery. 1st Edition., Addison-Wesley,
UK
-
The Royal Academy of Engineering (2004)
The Challenges of Complex IT Projects, ISBN 1-903496-15-2, April 2004
|
