|
1. Introduction
Our current information society is
based on the premise that the use of electronic information will bring
improvement in the quality of life of people (Japanese Government,
2000). Information technologies pervade almost every aspect of daily
life. New advances in software and telecommunications lead organisations
to invest considerable resources in integrating technology to their
operations. Because of their pervasive nature, evaluation of the impacts
of technology in society is important. Approaches to evaluation
increasingly focus on the inclusion of a variety of issues which affect
or may be affected by the implementation of information systems.
Progress has also been made in developing participative approaches in
which different stakeholders can raise issues related to the value that
they attribute to information systems (Remenyi and Sherwood-Smith,
1999). However, little has been said about the importance of
understanding the context of relations in which information systems are
implemented and evaluated as an enabler or facilitator of change.
In this paper, we develop a view of
the process of evaluation of information systems plans from the
perspective of power relations and their connections with the ethics of
the process. Michel Foucault’s ideas on power and ethics provide an
alternative understanding of evaluation as a process of continuous
tension between discourses about ethics. Using these ideas, we propose
to consider a critical view of evaluation processes as power strategies
that we must be aware of to achieve change in organisations and beyond
them. We provide an account of a practical evaluation exercise in which
one of us has been recently involved in a university setting. The
insights gained from this experience lead us to suggest some strategies
that could be used to rethink the scope of evaluation processes in
organisations. With these insights, we intend to contribute to the
ongoing debate about the development of participative approaches to
information systems evaluation.
The paper is structured as follows.
First, we provide a description of some issues which are emerging in the
field of information systems evaluation. We introduce the ideas of
Michel Foucault on power and ethics, and describe some of their
implications for the practice of information systems evaluation. Then we
present an experience of doing a short evaluation of an information
systems plan and emerging projects. We discuss the influence of
Foucault’s ideas in informing the role of evaluators and their decisions
during the process. Finally we present some insights for the evaluation
of information systems that could contribute to develop future research
in this area.
2. A context for evaluation
Currently, the notion of an
information society as one in which almost every activity involves
information (Information Society Commission, 1996) is manifested in a
diversity of information societies. This is partly due to the diversity
of actors that are directly or indirectly involved in defining or using
technologies, conditions of the market, and needs of local groups (Mansell
and Steinmueller, 2000). This diversity highlights the importance of
considering particular circumstances which affect the definition,
implementation and evaluation of information systems.
There is an emerging interest in
the development of continuous and participative approaches to evaluation
which could help the delivery of information systems maximising their
value to businesses (Remenyi and Sherwood-Smith, 1999). This is
complemented by the inclusion of different organisational issues whose
treatment could help to maximise the delivery of benefits by information
systems (Doherty and King, 2001). If the scope of evaluation is becoming
continuous, participative and integrated into the daily activities of an
organisation, evaluation practitioners need to understand the context of
relations between people as influencing and being influenced by the
process of evaluation. This requires a more comprehensive view of how
information systems produce a variety of effects in people’s relations,
and also how these relations shape the adoption of technology. In this
regard, this could also involve an understanding of power in evaluating
information systems.
3. Evaluation and power
Following Gregory (2000), our
analysis of power in evaluation is focused on participative approaches
to evaluation. The notion of power in evaluation is a great
unmentionable issue in evaluation theory (Gregory, 2000). Participative
approaches for evaluation stress the importance of involving different
groups of stakeholders in defining, carrying out and assessing the
outcomes of evaluation. This could also help to motivate people, to
empower them in taking control of the tasks of the evaluation process,
and ensure their commitment. Participation is seen as a way of
establishing open dialogue with those involved and affected in a
situation. In the realm of information systems, participation could help
to strengthen decision-making processes related to definition, design
and implementation of information requirements (Flynn, 1992).
In evaluating information systems,
participation and involvement of stakeholders has become an interesting
if not essential feature of formative approaches to evaluation (Doherty
and McAulay, 2001). Participation can contribute to learning from
experience and developing contingency plans to address emerging issues
which might be affecting the success of IS projects and therefore their
delivery of value to the business (McAulay, Keval and Doherty,
2001;Remenyi and Sherwood-Smith, 1999). More comprehensive approaches
that take into account a variety of organisational issues affecting
success in implementing information systems have been developed. Using
participative approaches could help identify these issues, and treat
them adequately to ensure that different impacts of an information
system (individual, social and economical) are desirable (Doherty and
King, 2001).
We take the view that the treatment
of these issues could be improved by reflecting on the assumptions made
about participation in evaluation, and how the treatment of
organisational issues could be challenged or improved from a perspective
based on power. As Gregory (2000) has argued, the use of participative
approaches for evaluation has provided little guidance on how to account
for the existence of power relations. Moreover, the power of some
stakeholders might affect and be affected by process and outcomes of
evaluation (Serafeimidis and Smithson, 1999) It has been accepted that
evaluation is a specific effort (i.e. project), which is associated to
another one (IS project). Evaluation should provide insights because of
the need of information systems to deliver value to organisations by
correcting the course of actions in projects (Walsham, 1993). This
indeed is setting an agenda influences the purposes of evaluation. The
risk is that, without a deeper degree of reflection, some inequalities
in power could be reinforced and perpetuated some of which might even be
contributing to the failure of projects despite an apparent success in
the short-term.
Furthermore, the treatment of a
wider variety of issues in evaluation could contribute to the delivery
of benefits that match and enhance organisational capabilities and
constraints (Doherty and King, 1998). This can be seen as finding or
developing ways in which the processes related to the implementation of
information systems integrate into the relations between people, and
contribute to achieve collective benefits (including improving
organisational performance). In this respect, a more detailed analysis
of how to identify and use existing relations could be useful to create
an environment of acceptance to the use of information systems.
4. Foucault’s research project
It is difficult to provide a
summary of Michel Foucault’s work on the history of Western civilisation
without missing or misunderstanding important issues. Foucault has
provided an interesting insight for the problem of the human subject, be
it individual or collective. For Foucault, the main question in modern
society is how human beings are constituted as subjects (Foucault,
1982;1982). The meaning of ‘subject’ is twofold: “someone subject to
someone else by control and dependence, and tied to his own identity by
a conscience or self-knowledge” (Foucault, 1982). Both meanings in the
above definition suggest a form of power, which subjugates and makes
subject to.
In his historical analysis of the
different forms of constitution of subjects, Foucault employs the
concept of power as an aide for analysis, which enables him to describe
how these forms have emerged and evolved. For Foucault power is
constituted by the relations between subjects and the same time, it
constitutes them. Power can be seen as a “total structure of actions
brought to bear upon possible actions: in incites, it induces, it
seduces, it makes easier or difficult” (Foucault, 1982). Power is
dynamic, elusive; it operates at different levels, and targets
individuals as well as collectives. Power influences, seduces; it makes
difficult or easier to do something (Foucault, 1977).
Distinct forms of power relations
interact. They mutually cancel, reinforce, contribute to or struggle
against each other. They are connected to forms of knowledge, which
produce objects to be known about; these objects also include the human
subjects. Therefore, relations of power also include relations between
subjects and themselves (Foucault, 1977;1984;1984). They contribute to
constitute what subjects are, what they know, how they act and behave
ethically and in relation to what they consider is right to be or to do.
Foucault’s notion of power brings
an essential connection between power and knowledge but also to our
understanding of ethics. Foucault argues that forms of power create
relations between subjects, themselves and others, which have
constrained them in their possibilities for action (Foucault, 1984). In
reacting against the potential risk of being ‘normalised’ (e.g. made
normal for the benefit of ‘society’ as constituted in relation to power
struggles), subjects could consider the possibilities of “no longer
being, doing or thinking what we are, do, or think” (Foucault, 1984
p.46). For Foucault, these possibilities need to be understood in the
presence of power and how it manifests itself in relations between
subjects according to historical and contingent conditions (Foucault,
1982;Kendall and Wickham, 1999). Subjects cannot escape from power, nor
does it only produce intended consequences. However people could
identify different options of action in relation to it. The power-ethics
relationship should not be understood only as a limiting one, but one
which sets responsibilities for managing our freedoms more consciously
(reference).
For the practice of participative
evaluation in information systems, these ideas suggest the possibility
of enhancing self-reflection between people about their own situation,
how knowledge is constituted or could be redefined. Strategies to
encourage participation could be assessed as strategies of power, which
need to take into account forms of power that they could reinforce,
resist or act upon. Furthermore, to those facilitating the evaluation,
an analysis of power and ethics could lead them to reflect on their own
identities. They could identify and explore the different roles that
make them acceptable (or rejected) in a particular context of
intervention. An adequate starting point for such analysis is in
situations where there is struggle to protect or promote different forms
of individualisation (Foucault, 1982). This is the case for instance
where there is the potential (or danger) for standardising the ways of
behaving, acting or thinking of people as ‘normal’ or ‘good’. Examples
of such analysis can be found elsewhere (Atkinson, 1998;Kendall and
Wickham, 1999). In the field of information systems, understanding the
potential and danger for standardising the ways of behaving could help
us to identify areas of creative improvement. For example, the tension
between the implementation of common technological software platforms
against moving forward via individualisation could be used as starting
point for the analysis of power relations.
A further reflection on power might
also lead practitioners and participants in evaluation to reflect on the
purposes of the process in relation to their own purposes as ethical
subjects, and act accordingly. This does not only mean that power can be
only resisted, but instead used strategically. Those who are evaluating
and being evaluated could develop their own agendas while still being in
evaluation, as a way of operating within the constraints and
possibilities for action offered by existing power relations.
5. An example
In this section we give a summary
of a first attempt to use these ideas in the practice of evaluation. A
full description of this experience is out of the scope of this paper.
Additionally, an analysis of how power operates in practice is
contingent to the conditions of the process, let alone to what we could
draw from it. The reflections are made after the exercise took place.
They are related to the process of evaluation developed as being
inspired by the ideas on power and ethics above presented.
In 2003, one of us (José) had the
opportunity to conduct an evaluation of an information systems planning
project which he had been involved in at Javeriana University in
Colombia in 1999 (Córdoba and Midgley, 2003). The purpose of this
project was to develop a series of understandings about the roles of
information technology at the institution that could help people to
improve the ways in which education was delivered. The project concluded
with a set of suggestions to improve current situation of the use of
information technology that were delivered to senior management. A
two-week project was proposed in 2003 to assess the extent to which
these suggestions had been taken on board by decision makers. The
purpose was also to learn about the evolution of information technology
during the period of four years that followed the process of planning.
Initially, the process was going to involve five people who had been
taken part in the project.
The main suggestions made in 1999
included: 1. Improve the coordination between IS projects and the
evaluation of the social impacts that information systems were to have.
2. Develop an information system to provide flexibility and support in
the delivery of lectures and in the definition of research projects in
conjunction with the industry. 3. Encourage members of the institution
to develop a culture of solidarity in which there could be tolerance and
respect for diversity (Córdoba and Midgley, 2003). At the time of being
presented, these suggestions were received with enthusiasm by senior
research managers. They were also included in discussions for the
definition of plans by the computer science department, where the
project had been based. Nevertheless, some of the participants in the
project were concerned about the implementation of the suggestions.
There was also the feeling that a participative process had been
creating expectations amongst the people that could not be satisfied,
and that the suggestions could be going against established plans. Those
who took part were instinctively recognising the potential tensions
between local departments and the wider institutional environment. We
will describe these tensions in more detail later on in this paper.
In order to conduct the evaluation,
José contacted the current head of the computer science department. José
initially aimed at getting the acceptance about the project, and know
how it was perceived. He advanced the idea that this project could
contribute to advance research on evaluation of information systems at
the institution. He also offered to help as a visiting (foreign)
researcher, so that people saw this as an opportunity to have a useful
resource for their research.
After a few weeks of proposing the
project, José received a response from one of the department managers.
They liked the idea, however they said that they were too busy to help
with the organisation of the research. Additionally they said that ‘the
department had changed a great deal’, therefore research in this area
was difficult to develop. At this point, it seemed that there was a need
to develop a strategy to be recognised as a valuable researcher in order
to get access to people in the institution. By having a few
conversations and exchanging some e-mails, José let the five
participants of the project in 1999 know about the possibility of doing
an evaluation. He could get hold of three of them; the other two never
replied. Those who replied seemed to be implicitly interested in getting
any new knowledge on research that José could offer to the institution.
He accepted this role and then organised a schedule for interviews. He
then went back to the computer science department with the news that he
had already organised some activities, which they offered to host for
him. Although this situation did not seem to be entirely consistent with
the initial purpose of the evaluation exercise, the role of a ‘foreign
expert’ seemed to be more accepted within the existing dynamics of power
at the institution. This role was connected to the delivery of
‘expert-based’ knowledge that was seen as valuable for some individuals
within the organisation.
On arrival to the university, José
decided to assume this role of ‘foreign expert’ in order to gain the
recognition of staff members and students, and to get to talk to those
who had been involved in 1999 about their perceptions of the project. He
was introduced to some people to whom he took the opportunity to explain
the project. By being based at the computer science department, José
also became involved in conversations with staff members. Some of them
were interested in finding out information about possibilities of
studying or doing research abroad, or how universities were developing
their electronic information services. In these conversations, José was
also able to exchange views about how people saw the evolution of the
role(s) of technology and how the suggestions made in 1999 had
contributed to it. At the time of evaluation, Javeriana was facing the
challenge to become internationally accredited, and this seemed to be
the main focus of attention of staff members inside and outside the
department. As an evaluator, taking the role of a ‘foreign expert’ was
shaping the role and the conversations that were possible, and the
topics to which this role was leading the evaluator to be engaged with.
José offered his help to facilitate debate in the meetings that were
organised to talk about the international accreditation of the
department. He was told that the format was already defined and there
was no need for his help.
In the interviews that he had
organised from before, José used a semi-structured questionnaire to
start the conversation, and get gradually to know and to get the feeling
of the main factors that could have contributed to the implementation or
failure of the suggestions made in 1999 (see appendix 1). With two of
the senior managers and a member of staff, José was asked about his main
interest in doing this evaluation exercise, as well as his current
affiliation with the institution. People seemed to want to know on which
side (for or against) José was standing on. The role of a foreign expert
seemed to be shaping the knowledge José was granted access to. This role
granted him access to some relations with people, and it was highlighted
that those taking part in conversations wanted to know what power José
had.
In one interview with a senior
research manager in which the conversation was brought around to the
suggestions made in 1999 to improve the situation, José was challenged
on the grounds that these suggestions seemed to be developed without
considering the scheme that had been designed for the assessment of
suggestions derived from research projects. This assessment scheme
stated that suggestions should be justified by departments, then taken
to the corresponding research committees and approved by faculty
directors. After this process was developed successfully (and perhaps
more justifiable suggestions were made), the senior management could
decide to allocate resources for their implementation, which in some
cases required further analysis. The scheme for discussing and selecting
was not developed until 2001. Although the suggestions made in 1999 did
not conform to that selection scheme, they were described as still
relevant for the current situation. Any proposal for change needed to be
framed within a particular discourse about what constituted a good
research methodology. For one of the senior managers, these suggestions
lacked credibility, as they had been the result of using a qualitative
methodology. For this manager, in order to properly justify any
suggestion, a quantitative survey should be done, and the suggestion
should be raised by a representative sample of the population at the
university.
Aside from a particular role which
was recognised as important, it could be said that there was a set of
practices (schemes) which needed to be used in order to produce adequate
or appropriate research, e.g. ethically acceptable. These practices
seemed to be embraced in a type of ‘institutional layer’ of knowledge
and power about information systems related projects. In this sense, the
idea of power is being used in a way consistent with Foucault: Actions
influencing other actions, projects and decisions about them influencing
others. Knowledge involves knowledge of how to justify projects
according to what is being regarded as acceptable to do: Knowledge of
methods used, schemes for approvals, reasons for the need of a project,
meetings in which it is advisable to present ideas, etc. The practices
discussed were complemented by some initiatives which were generated by
a proper process of justification (as described above) or the definition
of senior management initiatives. One of these projects was the
implementation of a corporate information system to support academic
registration. This project had been organised under the direct
supervision of senior management, and had been allocated management
resources, which gave it recognition and support within the institution.
This speaks about the nature of the
justification for a project and the relationships that exist between
typically used justification methods and the institutional layer
reinforcing existing power-ethics structures. Justification methods are
a key power strategy. In the case of the senior research manager,
particular forms of justification were used to profile projects, to
channel activities and resources, to reinforce his existing plans.
The operation of this institutional
layer has contributed to the implementation of some projects, and also
affected the success of others. One example of this situation can be
seen in the existence of a project which aim was to bring the use of a
new (virtual reality) technology at the university. The development of
this project had gone through different stages, also characterised by
the change in project leader role several times. The project has been
continuously re-started and one of its main activities is the gathering
of information about the existing use of this technology in departments
and faculties. The idea is that with this identification, the best
practice is selected and implemented across different areas. However,
timing in having a clear institutional policy has affected the selection
of any practice and its subsequent support for implementation. Those
leading the project were still required to continue with it.
Taking advantage of having a
physical space at the computer science department, José also organised
another two additional interviews: one with a lecturer at this
department, and another with the leader of an institutional project. The
latter was suggested by the head of the computer science department, as
it seemed that the project that this person was leading had strong
similarities with the one conducted in 1999. This opened up a new ground
of discussion about the development of information systems projects.
Parallel to the existence of an institutional layer, there seemed to be
‘local layer’ of initiatives embedded: It was composed of a series of
initiatives that had been developed to address the needs that certain
departments and areas had, which could be addressed by using information
technology. The initiatives developed their own justification that was
accepted at the level that provided them with resources for their
completion (i.e. faculty or department). Some of them had been keeping a
low profile in order not to get the interest of other areas or the same
institutional layer, and therefore their scope and the relations
developed between those involved did not transcend the boundaries of an
area or department. These initiatives could be potentially recognised as
having impacts across different areas and departments at the
institution. However, this required a change in their justification,
organisation and impact, and possibly their delay or cancellation when
being regarded as institutional projects. This possibility was not made
strongly, as a way of avoiding the potential for tension with the
institutional layer.
With respect to the justification
of projects at the local level, we can consider two illustrations of
‘local layer’ projects whose description emerged from interviews across
the organisation. The first was an application to support the monitoring
of progress of students. The second was a project to define possible
roles for technology in education. Each project is in a different
department. These two illustrate similarities in the way justification
works in the local layer, and the potential for tension between them and
the ‘institutional layer’. The similarities lie in the conception of a
local need, and in that the philosophical style reflects the
intellectual style of the group, and finally that the resources and
expertise are already under their command. In the case of the first
project local need came about because of large numbers of students, it
was led by the computer science engineering department team who were
driven by technical credibility. In the case of the second project it
was led by the need of the ‘open education unit’ who saw the necessity
to integrate technology with the educational process. The differences
between the projects demonstrate the potential but not certainty for the
local layer to be in tension with the institutional layer. The second
project was producing conceptual definitions placing an obligation to
the institutional layer to select particular projects. Additionally,
there were demands to change the course of some current institutional
projects. However, the institutional layer had its own justification
mechanisms that would not necessarily promote the same suggestions.
Furthermore, the institutional layer had ways of issuing policies and
definitions that had some conflict with the recommendations made by the
second project. Part of the tension was the lack of institutional
definitions for the use of technology.
From these two examples, it can be
seen that from the local layer may emerge projects consistent with those
who appeal to the organisational layer, but also those which are
potentially subversive to the organisational layer.
Between the institutional and local
layers there appeared to be a continuous power tension. Two of the
interviewees recalled the discomfort that some institutional initiatives
created among departments, and the lack of institutional guidance about
what to do with initiatives that were led by departments. Moreover, the
focus on achieving international accreditation seemed to lower the
priority of some of these initiatives. The tension was reinforced by the
development of a common practice between institutional and local levels.
Those initiatives with a potential institutional scope were required to
gather information across areas or departments about their own needs or
projects, as a way of ensuring that the most appropriate initiative
(that could satisfy most of the needs) was to receive institutional
support for its implementation.
José concluded the evaluation
exercise by conducting a workshop on evaluation of the central library
with staff members and users of it (see appendix 2). This enabled him to
contribute to improving the implementation of one of the institutional
projects (and therefore align himself with the institutional layer). At
the same time it opened up the opportunity to assume a more critical
role by posing questions about what could constitute good evaluation,
who else should be involved, and what other services could be offered to
different stakeholders. The questions were received with a mixture of
openness and scepticism, as they seemed to partially challenge the
self-image that the library had of being the most successful and
competitive area at the university.
José also presented a final report
to the computer science department and to those involved in the process.
In the report, the identification of the layers described above was
highlighted. Little response or comment has been received, which could
be partly due as the result of the engagement of people (including
researchers) in existing and new forms of power which enable them to
address their own ethical concerns.
6. Discussion
In this section we provide further
detail on our understanding of the notions of ethics and power as seen
in practice above. The use of these notions has enabled those involved
in conducting an evaluation to be aware of different situations that
emerge as the result of involving different types of participants in the
evaluation of an information systems plan. In the above situation, the
notion of power helped to understand how those facilitating the
evaluation (evaluators) are immersed in a set of practices and
justifications that render particular types of knowledge as true and
accepted (appropriate). It also enabled the identification of a variety
of relations between people by which evaluators can gain access to
sources of knowledge. These relations offer possibilities and
limitations for action.
A particular role for evaluation
(that of a ‘foreign expert’) led José (the evaluator) to engage in
certain relations, to get recognised as being part of certain dynamics
in which expertise was seen as valuable. The role of expert seemed
appropriate to establish relations with those in formal positions of
power, and collaborate with their purposes in the institution of
improving the delivery of information services and their
competitiveness. Assuming a more critical role to challenge the benefits
that information systems are giving to their users became a more
difficult task. When facing such difficulties, it was important that the
evaluator reflected on what type of ethical subject he wanted to be. The
evaluator had to be flexible to give some people what they needed to
know, in order to be able to engage in conversations in which he could
then challenge them.
In some of these conversations, a
variety of issues could be identified (i.e. lack of institutional
guidance, pressure to gain international accreditation) and this led to
identify an ‘apparatus of power’ that operated at two levels:
institutional and local. Each level had their own way of justifying
initiatives, putting them in place and implementing them. Each layer had
its own apparatus of power configured to reinforce their own
power-ethics. Implicitly, the institutional layer intends to shape the
practice of the local layer. The local layer may not intend to impact on
the other layer but may generate effects unintentionally. The evaluator
found himself as engaged in both levels, something that perhaps he could
do as an outsider to the dynamics of the institution. Due to the
tensions perceived, the evaluator needed to align at certain times with
one of these levels, and be careful not to ‘upset’ any of those people
who engaged him in any of them. However, the choices made (i.e.
inquiring about the suggestions of 1999, or presenting a final report)
had consequences that could not be fully foreseen, but which flow from
the desire of the evaluator to obey his/her own ethics. It can be argued
that in evaluation, any choice can be conceived of as the product of
power, and defines continuously new opportunities and constraints for
action, influencing the outcomes of any evaluation process and the
ethical identity of those involved.
With power and ethics, a particular
set of power relations were identified, which had some activities
associated with them. When the evaluator decided to conduct the
evaluation, he became inevitably immersed in power relations, to their
possibilities and constraints for action. The activities of evaluation
provoked a shift in these relations by introducing some challenges to
the nature and outcomes of information systems projects. At the same
time, they were also challenged and transformed by the same relations,
because some of those involved (including the researcher) defined some
opportunities for action (i.e. workshop on evaluation) or were denied
others (i.e. further conversations). In this regard, a more detailed
analysis of the dynamics of power could help to identify and raise
issues which affect the outcomes of evaluation projects, and inform
practitioners about different possibilities for action in relation to
what is possible and ethical to do.
Overall, the use of Michel
Foucault’s power-ethics concepts create the conditions for reflecting
and informing evaluation of information systems, as based on the
potential opportunities to deliver knowledge that could be made
acceptable among those involved in it. These situations of power-ethics
happen anyway. Our evaluation practices can be blind to them and
therefore poorer, or aware and critical of them, and therefore be more
robust. These ideas can help ensuring the usefulness of evaluation. This
action is informed by what those involved (including the evaluator)
consider is ethical to do, and this could cause tension in an evaluation
exercise. A more detailed analysis of how power operates in contexts of
evaluation is needed, which could contribute to the definition of
approaches, tools and methods to the improvement of practice in the
realm of information systems evaluation.
7. Concluding remarks
In this paper, we have used Michel
Foucault's ideas on power and ethics to enrich the practice of
participative evaluation of information systems. These ideas suggest the
need to be continuously aware of the constraints and opportunities for
action that any process of evaluation could have in the particular
context in which it takes place. They also raise the importance of
reflecting continuously about what it means to be ethical in evaluation,
and how this is transformed into meaningful action, including the design
of the research (participants, roles, purpose).
These ideas have been used to
inform an exercise of information systems evaluation related to a plan
already developed in a university. They informed the role(s) of the
evaluators to enable them to be flexible according to emerging
circumstances. They also supported the adoption of contingent strategies
for action, which were defined to operate within an environment
characterised by the existence of two layers of power (institutional and
local). In this particular case, the tensions between these layers
influenced the definition and implementation of information systems
related projects and therefore the evolution of information technology
in the organisation.
In our current information-based
society, the notions of power and ethics presented in this paper could
help practitioners to deal with the complexities encountered, and to be
aware of the dynamic nature of relations between people which influences
the deployment of information systems projects in organisations. The
development of a map of existing layers of power and consideration of
the strategies must form the basis of a reflective evaluation exercise.
Such maps could be personal or shared. This could contribute to a more
insightful evaluation practice. Further research could provide more
concrete methods and tools for analysing and dealing appropriately with
the dynamics of power and its connections with knowledge and ethics. The
analysis of power layers developed in the paper could inform future
analysis of similar type in evaluating information systems. The
following considerations could contribute to improve the development of
approaches which aim at improvement the treatment of a variety of issues
in the evaluation of information systems projects:
§ Understanding the wider
context in which power is identified and deployed. This involves
analysing the forces and factors (internal, external to the
organisation) which influence the adoption of particular power
strategies (i.e. issues which lead to the adoption of competitive
strategies).
§ Analysis of different
layers of power relations between people through which projects are
supported, justified, approved, or cancelled, delayed.
§ A reflective practice
about what is to be a 'good' evaluator, or a 'good' participant in
evaluation in relation to power relations.
8. Acknowledgment
We would like to thank all the
people at Javeriana University in Colombia who were involved in this
exercise, — particularly the staff members at the computers science and
systems engineering department who provided us with so much support.
9. Appendix 1: Questions for Interview
§ If you could describe
your experiences during the last 4 years, what is your view about the
evolution of the role(s) of information technology in the organisation?
(This might also include the delivery of information services).
§ Can you remember what
happened with the project? What were your positive and negative
experiences with the project in general (activities followed, methods of
research, the role of the facilitators, the suggestions made, etc)?
§ Looking at the
suggestions that we made at that time (the interviewer can show them to
the interviewee), which factors do you believe inhibited or enabled
their implementation?
§ If we had the
opportunity of conducting a similar project (e.g. a participative
process to define what is or are the roles of information technologies
in the organisation), what factors do you consider essential to address
and to avoid the mistakes that occurred in 1999?
§ How does the evaluation
of information technology projects and/or services take place in the
organisation nowadays?
This last question encourages the
interviewer and interviewee to reflect on the role of evaluation in the
organisation. After, the interviewer could proceed to find out more
about the mechanisms used for evaluation in an organisation, the
relationship between them and mechanisms of power used to confirm or
deny results from planning or evaluation, and the existence of different
forms of ethics that contribute to shape the behaviour of those involved
in an organisation.
10. Appendix 2. Workshop on Evaluation of
Information Services
The aim is to help those involved
in delivering and using electronic information services to identify some
opportunities to improve their evaluation mechanisms and activities. The
workshop includes the following activities:
§ Introduction.
Presentation of participants. A group of staff members and users
(students, academics) is recommended.
§ Brief presentation about
evaluation of services. The importance of evaluation is outlined.
Evaluation means finding out the value that users attribute to services,
hence the need to evaluate continuously and put in practice mechanisms
to make evaluation a daily activity.
§ Definition of services
and their attributes. Each service is seen as a ‘system’ which
transforms the needs of users in services with particular attributes.
The definition of attributes could be complemented by describing what
type of indicators might help those delivering the service with the
desired attributes.
§ Definition of evaluation
activities. The group defines a list of questions which could help to
validate the attributes gathered or to obtain more information to
improve the existing delivery. The questions can also be used to inform
the definition of activities of evaluation and some performance
indicators.
References
- Atkinson D 'The production of
the pupil as a subject within the art curriculum' Journal of
curriculum studies, Vol 30 No 1, (1998) pp 27-42.
- Córdoba J and Midgley G
'Addressing organisational and societal concerns: An application of
critical systems thinking to information systems planning in Colombia'
in Critical reflections on information systems: A systemic approach,
Cano J (ed s), Idea Group Publishing, Hershey (PA), (2003).
- Doherty N and King M 'An
investigation of the factors affecting the successful treatment of
organisational issues in systems development' European Journal of
Information Systems, Vol 10, (2001) pp 147-160.
- Doherty N and McAulay L The
relationship between the ex ante and ex post information systems
evaluation: Reflections from the literature and the practice of
evaluating e-commerce investments, MCIL, Paris, (2001).
- Flynn D J Information systems
requirements: Determination and analysis, McGraw-Hill, London, (1992).
- Foucault M The history of
sexuality volume one: The will to knowledge, Penguin, London, (1977).
- Foucault M 'Afterword: The
subject and power.' in Michel Foucault: Beyond Structuralism and
Hermeneutics., Dreyfus H and Rabinow P (ed s), The Harvester Press,
Brighton, (1982).
- Foucault M 'On the genealogy of
ethics: An overview of work in progress' in The Foucault Reader: An
Introduction to Foucault's Thought, Rabinow P (ed s), Penguin, London,
(1982).
- Foucault M The history of
sexuality volume two: The use of pleasure, Penguin, London, (1984).
- Foucault M 'What is
enlightenment? (unpublished French manuscript)' in The Foucault
Reader: An Introduction to Foucault's Thought, Rabinow P (ed s),
Penguin, London, (1984).
- Gregory A 'Problematizing
participation: A critical review of approaches to participation in
evaluation theory' Evaluation, Vol 6 No 2, (2000) pp 179-199.
- Information Society Commission
Information society Ireland: Strategy for action., Information Society
Commission Ireland, Dublin, (1996).
- Japanese Government Okinawa
Charter on Global Information Society, Declaration of the G8.,
Okinawa, (2000).
- Kendall G and Wickham G Using
Foucault's methods, Sage, London, (1999).
- Mansell R and Steinmueller W
Mobilizing the information society: Strategies for growth and
opportunity, Oxford University Press, Oxford, (2000).
- McAulay L, Keval N and Doherty N
The stakeholder dimension in information systems evaluation'', MCIL,
Oxford, (2001).
- Remenyi D and Sherwood-Smith M
'Maximise information systems value by continuous participative
evaluation' Logistics information management, Vol 12 No 1/2, (1999) pp
145-156.
- Serafeimidis V and Smithson S
'Rethinking the approaches to information systems investment
evaluation' Logistics information management, Vol 12 No 1-2, (1999) pp
94.
-
Walsham G Interpreting information systems in
organisations, John Wiley and Sons, Chichester, (1993).
|
