The Electronic Journal of Information Systems Evaluation provides critical perspectives on topics relevant to Information Systems Evaluation, with an emphasis on the organisational and management implications
For general enquiries email administrator@ejise.com
Click here to see other Scholarly Electronic Journals published by API
For a range of research text books on this and complimentary topics visit the Academic Bookshop

Information about the European Conference on Information Management and Evaluation is available here

linkedin-120 

twitter2-125 

fb_logo-125 

 

Journal Article

Post‑Implementation Evaluation of Collaborative Technology: a Case Study in Business Education  pp77-86

Andriani Piki

© Jan 2010 Volume 13 Issue 1, ECIME 2009, Editor: Elizabeth Frisk and Kerstin Grunden, pp1 - 96

Look inside Download PDF (free)

Abstract

To be successful in their future careers students need to develop diverse skills and qualifications. Firstly, in addition to understanding the course content and the underlying theories, students need to explore the implications that emerge from their practical application and develop their critical thinking and analytical skills. Secondly, students need to gain experience and confidence in working effectively within multidisciplinary and multicultural groups that mirror the situation they are likely to face in their future work environment. Thirdly, they need to familiarise themselves with collaborative technologies (CTs) since these are increasingly used in the workplace to facilitate communication and collaboration between distant co‑workers. To address these learning needs it is essential to incorporate CTs (such as videoconferencing systems) in the curriculum and provide well‑organized opportunities for students to gain hands‑on experience. Nevertheless, what technologies are used does not make the difference between motivated and unmotivated students; it is how these technologies are used that matters. Whilst innovative technologies can be fascinating, they must be properly evaluated and adjusted to specific educational, individual, and group needs in order to be successfully adopted by students. This evaluation entails taking into consideration the context within which the technology will be used (appropriateness evaluation) and the social‑psychological motives for user acceptance (evaluation of user satisfaction). This paper reports the findings from an interpretive case study in postgraduate business education where students were using a state‑of‑the‑art videoconferencing system as part of their workshops and group discussion sessions. This setting provided a suitable social milieu for post‑implementation evaluation of this collaborative technology. Qualitative methods were employed including participant observation, focus groups, and analysis of videoconferencing sessions captured on video. The findings indicate that computer‑supported collaborative learning (CSCL) helps students become confident with using CTs, learn best practices for communicating and collaborating effectively in technology‑mediated settings, and appreciate the impact that technology has on everyday social endeavours. The videoconferencing exercises also engaged students to actively participate in the learning process. Given the duality of technology presence (in educational and business contexts alike) the findings can inform the design of new pedagogical models that maximize the learning potential of CTs.

 

Keywords: computer-supported collaborative learning, CSCL, videoconferencing, collaborative technology, CT, business education, post-implementation evaluation, video-ethnography, case study

 

Share |

Journal Article

Evaluating the Benefits of Regional Electronic Marketplaces: Assessing the Quality of the REM Success Model  pp11-20

Denise E Gengatharen, Craig Standing

© Jan 2004 Volume 7 Issue 1, Editor: Dan Remenyi, pp1 - 66

Look inside Download PDF (free)

Abstract

A number of regional Internet electronic marketplaces (REMs) have failed or are floundering, partly due to the lack of proper evaluation of their costs and benefits. This paper uses a conceptual REM Success Model to examine the costs and benefits of a REM in Western Australia. The model has been derived from an extension to the Updated DeLone & McClean IS Success Model. The findings from the case study indicate that the REM Success Model, which includes cognisance of SME‑profile and motivation of the market maker, allows up‑front identification of the costs and benefits to all stakeholders.

 

Keywords: E-Commerce, Regional Electronic Marketplaces, Small and Medium Enterprises, SMEs, Evaluation of Benefits, REM Success Model

 

Share |

Journal Article

Developing an Evaluation Instrument for e‑Commerce Web Sites from the First‑Time Buyer's Viewpoint  pp31-42

Wei-Hsi Hung, Robert J McQueen

© Jan 2004 Volume 7 Issue 1, Editor: Dan Remenyi, pp1 - 66

Look inside Download PDF (free)

Abstract

This paper presents the process of developing an evaluation instrument specifically for the evaluation of e‑Commerce Web sites from the first‑time buyer's viewpoint. The development process is based on theoretical discussions of the Web evaluation and Web user satisfaction literature. A draft evaluation instrument was developed. To enhance its reliability and validity, several iterative trials on e‑Commerce Web sites were conducted. Some modifications were made to the instrument. The final version is capable of evaluating e‑ Commerce Web sites effectively. The instrument provides implications to both Web evaluation practitioners and academics.

 

Keywords: e-Commerce, Web evaluation, user satisfaction, transaction activity, instrument

 

Share |

Journal Article

A Chronic Wound Healing Information Technology System: Design, Testing, and Evaluation in Clinic  pp57-66

Antonio Sánchez

© Jan 2004 Volume 7 Issue 1, Editor: Dan Remenyi, pp1 - 66

Look inside Download PDF (free)

Abstract

In the UK, chronic wound healing is an area of specialist clinical medicine that operates within the framework of the National Health Service. It has been the basis for the design, testing and evaluation of a prototype system of information and communication technology (ICT), specifically adapted to the domain. Different wound healing clinics were examined using a combination of 'hard' and 'soft' methods to allow a richer perspective of the activity and gain a deeper understanding of the human activity, its relation to the working information system, the existing information technology (IT), and the potential of a comprehensive IT system to manipulate live data in clinic. Clinicians and administration staff were included in all aspects of the process to enhance the design lifecycle and the understanding of the process. An observe, report, plan and act (ORPA) cycle, based on the dictates of action research, was established to accomplish the design and testing of a system that clinicians were comfortable enough with to consider its use in clinic. Three different strategies were applied to evaluate its use in participating clinics. Cultural historical activity theory was used as the main framework to analyse the activity system, and to interpret the clinicians and the systems performance, as well as their evaluation of the experience. Activity breakdown areas are suggested and reasons for them are considered in the light of wound care workers feedback, and the researcher's observations, notes, and analysis.

 

Keywords: Electronic data manipulation, clinical ICT, information technology evaluation

 

Share |

Journal Article

Modelling Risks in ISIT Projects through Causal and Cognitive Mapping  pp1-10

Abdullah J. Al-Shehab, Robert T. Hughes, Graham Winstanley

© Jan 2005 Volume 8 Issue 1, Editor: Dan Remenyi, pp1 - 80

Look inside Download PDF (free)

Abstract

Software systems development and implementation have become more difficult with the rapid introduction of new technology and the increasing complexity of the marketplace. This paper proposes an evaluation framework for identifying the causes of shortfalls in implemented information system projects. This framework has been developed during a longitudinal case study of a problematic project, which is described.

 

Keywords: causal and cognitive mapping, project evaluation, information systems project risk

 

Share |

Journal Article

Peer Assessment: A Complementary Instrument to Recognise Individual Contributions in IS Student Group Projects  pp61-70

Elsje Scott, Nata van der Merwe, Derek Smith

© Jan 2005 Volume 8 Issue 1, Editor: Dan Remenyi, pp1 - 80

Look inside Download PDF (free)

Abstract

This paper discusses peer assessment as a component of the assessment strategy used for Information Systems student group projects at a South African university. The value of peer assessment and the contribution to the real‑life experience offered by group projects, will be discussed. It will also illustrate how this process adds value by enhancing deep learning. Its value as a complementary assessment instrument in a multiple assessment strategy and how the results of peer assessment are used to recognise individual contributions to group performance will be illustrated. The use of peer assessment as an instrument for both informal formative assessment and formal summative assessment will be described. To perform the peer assessment specific instruments were designed and used throughout the lifecycle of the course.

 

Keywords: Peer assessment, group work, assessment, self-assessment, IS Project

 

Share |

Journal Article

Broadening Information Systems Evaluation Through Narratives  pp115-122

Jonas Hedman, Andreas Borell

© Sep 2005 Volume 8 Issue 2, Editor: Dan Remenyi, pp81 - 142

Look inside Download PDF (free)

Abstract

The purpose of information systems post‑evaluation ought to be to improve the use of systems. The paper proposes the use of narratives as a tool in post‑evaluations. The potential in narratives is that they can convey meanings, interpretations, and knowledge about the system, which may potentially lead to action. The paper offer three main suggestions: 1) evaluations should form the basis for action; 2) narratives makes evaluation more relevant; and 3) post‑evaluations should be done with the aim of improving use. Narratives should be viewed as a complement to traditional evaluation methods and as a way of making evaluation more formative and thereby moving away from the more common summative perception of evaluation. The conclusion of the paper is that narratives can advance IS evaluation and provide a richer evaluation picture by conveying meanings not included in traditional evaluations.

 

Keywords: Narratives, information systems evaluation, measurements, measure, stories, action

 

Share |

Journal Article

Measuring the Quality of Electronic Journals  pp133-142

Maricela Lopez Ornelas, Graciela Cordero Arroyo, Eduardo Backhoff Escudero

© Sep 2005 Volume 8 Issue 2, Editor: Dan Remenyi, pp81 - 142

Look inside Download PDF (free)

Abstract

This paper presents the methodology developed to create a system to evaluate academic electronic journals. This methodology was developed in two stages. In the first stage, a system to evaluate electronic journals was created. The criteria framework and the indicators for assessment for academic electronic journals were selected and defined. According to this framework, several questions were designed to measure each indicator and, as a result, an instrument to evaluate academic electronic journals was built. In the second stage, this instrument was validated by 16 editors of electronic journals of different countries and different areas of knowledge that were considered as judges to evaluate clarity, importance, relevance and coverage of each question, indicator and criteria. This instrument was distributed by e‑mail. The opinions given by the judges were processed and then used to help in the construction of a new instrument that is ready to be presented to the Mexican Council of Scientific Research in order to evaluate Mexican academic electronic journals.

 

Keywords: Key words electronic journals, journals quality indicators, journals evaluation

 

Share |