The Electronic Journal of Information Systems Evaluation provides critical perspectives on topics relevant to Information Systems Evaluation, with an emphasis on the organisational and management implications
For general enquiries email administrator@ejise.com
Click here to see other Scholarly Electronic Journals published by API
For a range of research text books on this and complimentary topics visit the Academic Bookshop

Information about the European Conference on Information Management and Evaluation is available here

linkedin-120 

twitter2-125 

fb_logo-125 

 

Journal Article

Evaluating Information Systems according to Stakeholders: a pragmatic perspective and method  pp73-88

Jenny Lagsten

© Jan 2011 Volume 14 Issue 1, ECIME 2010 Special Issue, Editor: Miguel de Castro Neto, pp1 - 166

Look inside Download PDF (free)

Abstract

In the last decade several researchers have addressed the problem that there does not seem to be much evidence of extensive use of interpretive evaluation approaches in practice. Researchers have though recognized the interpretive evaluation approach as well founded academically and theoretically offering potential advantages such as stakeholder commitment and learning opportunities. One reason for this non‑use could be that there are few, if any, interpretive evaluation methods ready at hand for evaluators in practice. An interpretive IS evaluation method means a method in support for doing evaluation as interpretation. This research presents a practical method for doing evaluation of information systems as a joint act of interpretation performed by the stakeholders of the information system in use. In our research we have expanded the interpretive philosophical base to embrace a pragmatic knowledge interest in order to underpin the overall strive for evaluation that is to contribute to change and betterment. The method presented is named VISU (Swedish acronym for IS evaluation for workpractice development). The process of evaluating accordingly to the VISU method has been extensively tested in practice and in theoretical grounding processes and is now considered ready for wider use. The research process for developing VISU has been conducted with canonical action research through parallel work with evaluation and method development in six episodes within two cases. VISU consists of prescribed actions that are anchored in a set of underlying principles stemming from the philosophy of American pragmatism. Evaluation according to VISU is performed in three phases; arrange, evaluate and develop. In the paper VISU is described according to phases, actions, main concepts and principles. The use of VISU is demonstrated through examples from a performed evaluation of an information system in support for social welfare services.

 

Keywords: IS evaluation, stakeholder model, interpretive IS evaluation method, pragmatism, action research

 

Share |

Journal Article

Evaluating Enterprise Systems Implementation Methodologies in Action: Focusing Formalised and Situational Aspects  pp83-90

Daniela Mihailescu, Sven A. Carlsson, Marius Mihailescu

© Jan 2007 Volume 10 Issue 1, ECITE 2006 Special, Editor: Dan Remenyi, pp1 - 122

Look inside Download PDF (free)

Abstract

Enterprise Systems (ES) are often the largest and most important Information Systems (IS) an organisation employs. Most ES are rented or bought as COTS (Commercial Off The Shelf) software. The use of COTS leads to a distinction between the development of the ES software—done by ES software providers, like SAP and Microsoft—and the implementation of ES software in a specific organisation. Implementation of ES are often associated with problems like higher implementation cost and longer implementation process than anticipated. To improve ES implementation, ES providers increasingly support their ES software by, in part computer‑based, implementation methodologies. The paper present an ES implementation evaluation framework called ES Implementation Methodology‑in‑Action. The framework integrates two complementary views: 1) a technology view, focusing on the formalised aspects as expressed in the ES implementation methodology (the content of the methodology), and 2) a structural view, focusing situational aspects as expressed by the implementers (the users of the implementation methodology) including implementers, implementation context, ES software and other individuals participating in the implementation project. Using document studies and interviews with implementers we show how the framework can be used to evaluate ES implementation methodologies. We evaluate one well‑known ES implementation methodology: SAP's ASAP.

 

Keywords: Enterprise Systems Implementation Methodology, Evaluation Framework, Implementation Methodology in Action, Methodology Evaluation

 

Share |

Journal Article

Adding Action to the Information Audit  pp271-281

Huan Vo-Tran

© Sep 2011 Volume 14 Issue 2, ICIME 2011, Editor: Ken Grant, pp167 - 281

Look inside Download PDF (free)

Abstract

The Information Audit (IA) has long been seen as an important tool within the Information Management field, with its origins stemming from financial audits. It is used extensively in libraries as an improvement tool and, although many have tried to define it, such as Guy St. Clair (1997), Orna (1999) and Henczel (2001a), there is still no general consensus on a definition, or the steps taken to achieve it. Whatever form it may take, it is agreed that to undertake such a task requires a structured approach. The following study will propose a hybrid approach in which Henczels seven‑stage Information Audit model will be coupled with the Action Research (AR) methodology in order to assist a mid‑sized architectural practice to manage their information throughout the architectural design process, and, in particular, as they attempt to design a new academic building for a prominent Australian university.

 

Keywords: information audit, information management, architectural design process, action research

 

Share |

Journal Article

Developing an Evaluation Instrument for e‑Commerce Web Sites from the First‑Time Buyer's Viewpoint  pp31-42

Wei-Hsi Hung, Robert J McQueen

© Jan 2004 Volume 7 Issue 1, Editor: Dan Remenyi, pp1 - 66

Look inside Download PDF (free)

Abstract

This paper presents the process of developing an evaluation instrument specifically for the evaluation of e‑Commerce Web sites from the first‑time buyer's viewpoint. The development process is based on theoretical discussions of the Web evaluation and Web user satisfaction literature. A draft evaluation instrument was developed. To enhance its reliability and validity, several iterative trials on e‑Commerce Web sites were conducted. Some modifications were made to the instrument. The final version is capable of evaluating e‑ Commerce Web sites effectively. The instrument provides implications to both Web evaluation practitioners and academics.

 

Keywords: e-Commerce, Web evaluation, user satisfaction, transaction activity, instrument

 

Share |

Journal Article

Broadening Information Systems Evaluation Through Narratives  pp115-122

Jonas Hedman, Andreas Borell

© Sep 2005 Volume 8 Issue 2, Editor: Dan Remenyi, pp81 - 142

Look inside Download PDF (free)

Abstract

The purpose of information systems post‑evaluation ought to be to improve the use of systems. The paper proposes the use of narratives as a tool in post‑evaluations. The potential in narratives is that they can convey meanings, interpretations, and knowledge about the system, which may potentially lead to action. The paper offer three main suggestions: 1) evaluations should form the basis for action; 2) narratives makes evaluation more relevant; and 3) post‑evaluations should be done with the aim of improving use. Narratives should be viewed as a complement to traditional evaluation methods and as a way of making evaluation more formative and thereby moving away from the more common summative perception of evaluation. The conclusion of the paper is that narratives can advance IS evaluation and provide a richer evaluation picture by conveying meanings not included in traditional evaluations.

 

Keywords: Narratives, information systems evaluation, measurements, measure, stories, action

 

Share |

Journal Article

Testing of a Model Evaluating e‑Government Portal Acceptance and Satisfaction  pp35-46

Cora Sio Kuan Lai, Guilherme Pires

© Jan 2010 Volume 13 Issue 1, ECIME 2009, Editor: Elizabeth Frisk and Kerstin Grunden, pp1 - 96

Look inside Download PDF (free)

Abstract

e‑Government has the potential to improve public administration efficiency by increasing convenience, performance and accessibility of government information and service to users. But knowledge about e‑Government remains limited. To realize its potential, e‑Government needs to be grounded on in‑depth understanding of target users needs, perceptions and other factors influencing its uptake. This cross‑sectional study identifies and examines factors influencing e‑Government portal satisfaction and adoption by individual citizens in Macao, three years after its inauguration. It is an adaptation to the e‑Government context of a model developed for assessing e‑commerce websites. To understand the determinants of e‑government portal adoption, an integrated model of user satisfaction and technology acceptance is empirically tested. The integrated model involves four success factors: information quality, system quality, perceived effectiveness and social influence, which impact user satisfaction with the e‑Government website, influencing intention to reuse. Overall, the study proposes that user perceptions about the e‑Government portal influence user attitude towards the portal. An Internet survey collected data from 464 online users of Macao’s e‑government portal. The model was found to explain a large proportion of the variance in citizen’s intention to reuse the portal. The portal partially mediates the relationship between success factors and intention‑to‑reuse. The results provide evidence that Information Quality, System Quality and Social Influence (but not Perceived Effectiveness) are success factors influencing user satisfaction and adoption. It is recommended that portal management needs to ensure ease‑of‑use, currency and accuracy of the supplied information. Timely information updating is a major concern for the e‑Government portal in Macao. The content an e‑government portal that is perceived by users to be easier to navigate is likely to facilitate satisfaction and reuse. Finally, the importance of social influence justifies, managerial actions aimed at improving e‑Government portal acceptance by individual users and government employees.

 

Keywords: e-government portal, adoption, satisfaction, TAM, EUS

 

Share |

Journal Article

Where is Information Ethics in Iranian Library and Information Science Publications and Services?  pp89-94

Mortaza Kokabi

© Jan 2009 Volume 12 Issue 1, ECIME 2008, Editor: Dan Remenyi, pp1 - 118

Look inside Download PDF (free)

Abstract

There seems to be very few signs of the politeness peculiar to Iranians, when considering information ethics in Iranian library and information science scene. An expressed dissatisfaction appears to exist with library services among users studied in some dissertations on user satisfaction in library and information science in Iran. In spite of this fact that might be at least partly related to misbehavior of librarians, the words "ethics", "moral issues", and "morality" are not found in almost all of the publications related to library and information science in Iran, even in the most formal ones. These publications and documents as well as the current attempts to develop the topic in Iran, including both publications and activities, will be studied in this paper. The social, economic, and ethical aspects of the issue including the misunderstanding of the concepts of who serves and who must be served, who pays the tax and who must obtain service due to tax paying, the overall dissatisfaction of librarians concerning their social status as well as salaries and wages, the low costs, if any, of library and information services in Iran, will also be considered to show why this negligence has occurred in the profession. There are some efforts to be made however, to improve the situation. Library and information science educators firstly must do their best to show the significance of ethics in the profession. Their efforts must comprise the formal and informal teaching of information ethics to their students. Inclusion of courses of ethics in formal syllabi is among formal attempts. The demonstration of this ethics in LIS educator's behavior is an informal one. Publication of papers on information ethics is another duty of LIS educators. Setting up workshops on information ethics is a necessary step to be taken. Professional associations such as Iranian Library and Information Science Association (ILISA) can play an important role on the scene. The provision of an information code of ethics is a major responsibility of this association and is highly recommended.

 

Keywords: information ethics, Iran, user satisfaction, Iranian libraries

 

Share |

Journal Article

An Analysis of Three SERVQUAL Variations in Measuring Information System Service Quality  pp149-162

James J. Jiang, Gary Klein, Neeraj Parolia, Yuzhu Li

© Jan 2012 Volume 15 Issue 2, Editor: Shaun Pather, pp149 - 229

Look inside Download PDF (free)

Abstract

Service provided by the information system (IS) department within an organization has come to be considered a major component of IS success. The determination of service quality is considered as a comparison process between an expected level of service and the service perceived by the user. In past research, an IS adaptation of the SERVQUAL measure from the marketing literature was commonly used since it considers both the expectation and performance components of service quality. IS researchers have applied the IS SERVQUAL metric in various forms, including as a difference score, as a single component only, and as two distinct components. The choice of an IS SERVQUAL variation was usually made based on psychometric properties of the scale or explained variance. Few considered the implications that the chosen form of IS SERVQUAL variation has on the relationship between service quality and a dependent variable such as satisfaction or on the theoretical interpretation of the discrepancy theories from which service quality measure is derived. We examined the implications to research models and theory due to choosing the form based on statistical properties. The two component form holds truest to theory and still retains valued statistical properties that are important to researchers. The one component form that includes on performance considerations is still superior to the difference score model. For purposes of prediction more useful for practitioners, the single component and two component model greatly outperform the predictive ability of the difference score model, with the two component model being slightly better than the single component model.

 

Keywords: information systems, service quality, SERVQUAL, service performance, service expectations, difference scores, user satisfaction, quality evaluations

 

Share |