The Electronic Journal of Information Systems Evaluation provides critical perspectives on topics relevant to Information Systems Evaluation, with an emphasis on the organisational and management implications
Click here to see other Scholarly Electronic Journals published by API
For a range of research text books on this and complimentary topics visit the Academic Bookshop
Information about the International Conference on Information Management and Evaluation is available here

Information about the European Conference on Information Management and Evaluation is available here

To join the EJISE review committee click here

linkedin-120 

twitter2-125 

fb_logo-125 

 

Journal Article

A Process Capability Approach to Information Systems Effectiveness Evaluation  pp7-14

Sevgi Ozkan

© Mar 2006 Volume 9 Issue 1, Editor: Dan Remenyi, pp1 - 43

Look inside Download PDF (free)

Abstract

While defining or measuring the effectiveness of the information systems (IS) function has proven complicated, further effort on refining IS assessment is essential for the effective management and continuous improvement of both the IS function and the organisation. In addition, an effort to investigate the relationships among the established IS assessment tools to better reconcile their existing differences is warranted. This paper aims to clearly differentiate the notions of 'Software' from 'Information Systems'. A new IS assessment model is proposed to provide a more holistic view on how IS quality may be assessed by means of a process capability understanding of evaluating IS effectiveness within the organisational context.

 

Keywords: Information systems quality, Information systems effectiveness, Assessment, Software process maturity, Process capability

 

Share |

Journal Article

The Dilution of Effort in Self‑Evaluating Development Teams: Agile Loafing  pp175-186

John McAvoy, Tom Butler

© Feb 2010 Volume 12 Issue 2, Editor: Shaun Pather, pp129 - 198

Look inside Download PDF (free)

Abstract

Attempts to resolve the problems in software development have concentrated on the tools and methodologies used, despite an acceptance by many that it is a sociological problem. An example of this is the procedures and processes surrounding evaluations within projects, yet ultimately it depends on individuals more than process. This paper examines one of the sociological factors inherent in a software development team to determine its impact on evaluation within a project. Social loafing occurs where individual members of a team demonstrate a tendency not to work as hard as they could or should. This "slacking off" occurs because the team provides a degree of anonymity – the individual feels their lack of work will be hidden from evaluation within the overall output of the team. Some authors purport that Agile Software development teams have low incidences of social loafing (though these are opinions rather than research findings); the contrary can also be argued. An examination of the philosophy behind Agile Software Development, demonstrated by the Agile Manifesto, highlighted the possibility of occurrences of social loafing brought about by the Agile values. Agile espouses the importance of cohesive teams, the empowerment of these teams, and the collective ownership and self‑ evaluation of work by the team. These values map onto factors which are described as affecting social loafing. An investigation of two teams over an eight month period examined if the Agile values could lead to incidences of social loafing, specifically when their work is being evaluated The investigation determined that the opposite was actually the case. This paper then goes on to determine why the findings go against the initial hypothesis and to show the impact this can have on those evaluating software development projects.

 

Keywords: teams, agile software development, social loafing, self-evaluation, participant observation, sociological factors

 

Share |

Journal Article

Is a Multi‑Criteria Evaluation Tool Reserved for Experts?  pp151-162

C. Sanga, I. M Venter

© Feb 2010 Volume 12 Issue 2, Editor: Shaun Pather, pp129 - 198

Look inside Download PDF (free)

Abstract

The objective of this investigation was to determine whether the analytical hierarchy process algorithm is suitable for the evaluation of software by evaluators with little Information Technology experience. The scope of the research was the evaluation of two free and open source e‑learning systems at the Open University of Tanzania using 33 stakeholders with diverse levels of Information Technology experience. Both quantitative and qualitative research methods were used. The qualitative methods comprised participative observation and interviews. Questionnaires and the analytical hierarchy process, a multiple‑criteria decision‑ making algorithm, represented the quantitative methods. The results showed that of the two e‑learning systems evaluated, Moodle was preferred over ATutor. Furthermore it was found that the analytical hierarchy process algorithm is appropriate for the evaluation of software in a situation where Information Technology experience is limited. It is anticipated that the paper contributes to the theory and practise of decision making in developing countries such as Tanzania.

 

Keywords: free and open source software, e-learning systems, software quality, multi-criteria evaluation tool, analytical hierarchy process, novice user, developing country

 

Share |

Journal Article

Evaluation of Information Technology Productivity and Productive Efficiency in Australia  pp207-210

Wesley Shu, Simon Poon

© Jan 2006 Volume 8 Issue 3, ECITE 2005 Special, Editor: Dan Remenyi, pp143 - 230

Look inside Download PDF (free)

Abstract

The purpose of this paper is to evaluate IT productivity in Australia. Our model incorporates profit maximiza‑tion assumption and allows inefficiency. You have found IT hardware is the single factor which provides positive contribu‑ tion to the productivity.

 

Keywords: productivity, software, and productive efficiency

 

Share |

Journal Article

Causal Relationships between Improvements in Software Development Processes and Final Software Product Quality  pp1-10

Rini van Solingen, Egon Berghout

© Mar 2008 Volume 11 Issue 1, Editor: Dan Remenyi, pp1 - 51

Look inside Download PDF (free)

Abstract

A main assumption of software process improvement (SPI) is that improvements in a software development process result in higher quality software products. In other words, SPI assumes the existence of causal relations between process and product characteristics. To what extent have these causal relations, however, been explored? Which specific process improvements have which particular impact on which particular product quality attributes? In this paper an overview is given of these "software process and product dependencies" (PPD). This overview comprises of a list of SPI‑techniques and the associated product quality attributes that are addressed with these techniques. The extent of the causality is investigated and whether there is a possibility to identify more or less effective strategies for product quality improvement. The overview is based on a literature study and expert evaluation. The research is summarised in a matrix of both software process elements and associated software product quality characteristics. This matrix contains both satisfactory and unsatisfactory results. On the one hand, a promising extensive base of publications on techniques and methods was identified. On the other, a disappointing deficiency of empirical validation regarding the actual impact of those techniques on product quality is also prominent. As it is, we remain with an inadequate and incomplete indication of the product characteristics that particular software process improvement techniques intend to ameliorate. This article, therefore, hopefully, also provides a basis for discussion on the need to make process‑product dependencies more explicit.

 

Keywords: software development, software process improvement, learning, product-process dependencies, PPD

 

Share |

Journal Article

The Evaluation of Software Quality Factors in Very Large Information Systems  pp43-48

Souheil Khaddaj, G Horgan

© Jan 2004 Volume 7 Issue 1, Editor: Dan Remenyi, pp1 - 66

Look inside Download PDF (free)

Abstract

A quality model links together and defines the various software metrics and measurement techniques that an organisation uses which when measured, the approach taken must be sufficiently general for hybrid hardware and software systems. In this work software quality factors that should be taken into account in very large information systems will be considered. Such systems will require a high degree of parallelism and will involve a large number of processing elements. We start by identifying the metrics and measurement approaches that can be used. Many of the quality factors would be applied in similar way for sequential and paralleldistributed architectures, however a number of factors will be investigated which are relevant to the parallel class. In such a system many elements can fail which can have major impact on the system's performance, and therefore it affects the costbenefit factors. Portability and usability are other major problems that need to be taken into account when considering all the relevant factors that affect quality for such environments.

 

Keywords: Quality Modeling, Quality Measurement, Software Quality, Very Large Information Systems, Distributed Computing

 

Share |

Journal Article

Wiki‑Based Knowledge Management in a Transport Consultancy, a Case Study  pp133-142

Robbert in 't-Hout, Jos Vrancken, Pieter Schrijnen

© Oct 2010 Volume 13 Issue 2, ICIME 2010, Editor: Shaun Pather and Corrie Uys, pp97 - 196

Look inside Download PDF (free)

Abstract

Developing a Municipal Traffic and Transport Plan (MTTP) is a long lasting and complex process. Many different disciplines are involved, as well as many stakeholders. The process may take more than two years. The larger municipalities in the Netherlands mostly develop their own plans. But for the medium sized and smaller municipalities private consultancies play a major role in the development of the MTTPs. This article describes a case study concerning improving the MTTP development process in such a consultancy. The company did reasonably well in the field, but could do better: a part of the knowledge present in the company wasn’t shared well enough, especially the exchange of knowledge and experience between senior and junior consultants which needed improvement. To improve the sharing, a wiki was developed. In interaction with the consultants a structure was proposed, allowing them to add the information they valued relevant for the development of the MTTPs. The wiki appeared to reveal not only explicit knowledge, but also tacit knowledge. On top of that, the tacit knowledge often was personal, subjective, and even divergent. The juniors were more eager to work with the wiki than the seniors. There still exists a difference between the generations in their ability and readiness to use ICT tools. Sharing the tacit knowledge, revealing the subjective perspectives of the consultants was confronting the company with its implicit learning styles. The staff of the company believed they worked with objective knowledge, and that only such knowledge was relevant in planning and decision making. The wiki revealed that the subjective aspects actually played a role within the company. After some interactions with the manager of the department, the structure of the wiki was adapted. One part of the wiki was meant for explicit, objective knowledge; the other part for tacit, subjective knowledge. Arrangements were made to create the role of moderator for the themes in the wiki ‑ seniors that could decide how to deal with the subjective information. The conclusion of this case can be that wikis aren’t just neutral tools. They need to be tuned to the learning styles that are available within the community that will use the tool. Pilots can help in revealing the way organisations deal with knowledge management. The article follows the chronology of the project. It starts with a short picture of the process of developing an MTTP and with a summary of the learning theories used to assess the challenges of the company. The article proceeds with the design of the wiki and the introduction of the first layout. It then describes the reactions from the consultants and the manager, and the adaptation of the wiki. The article ends with some conclusions on the way wikis can be designed.

 

Keywords: municipal traffic and transport planning, wiki, social software, knowledge management

 

Share |

Journal Article

A Visualization‑based Approach to Present and Assess Technical Documentation Quality  pp150-159

Anna Wingkvist, Morgan Ericsson, Welf Lowe

© Jan 2011 Volume 14 Issue 1, ECIME 2010 Special Issue, Editor: Miguel de Castro Neto, pp1 - 166

Look inside Download PDF (free)

Abstract

Technical documentation has moved from printed booklets to electronic versions that need to be updated continuously to match product development and user demands. There is an imminent need to ensure the quality of technical documentation, i.e., information that follows a product. In order to ensure the quality of technical documentation, it is important to be able to measure it in a constructive way. In this paper, we approach quality from a software quality perspective and rely on automated measurements and analyses. It is generally not possible to assess quality attributes such as “ease of understanding” using automated measurements. To assess such quality attributes, we suggest the use of visualizations as a communication medium between the machine results and technical writers, and define a visualization‑based quality assessment approach for technical documentation. In order to test our approach, we use it to assess the quality of 3 real‑world documentations from a Swedish mobile phone manufacture, a Japanese camera manufacturer, and a Swedish warship producer. The study shows that our approach can be used to identify potential quality defects. For example, we tested an unclassified subset of the warship’s technical documentation and found that 49% of it was redundant text clones. We performed the study in collaboration with a Swedish company that is in charge of creating and maintaining the 3 documentations, and they acknowledge that our approach has great potential and that our results proved helpful to them.

 

Keywords: Information Quality, Software Analysis, Software Visualization, Technical Documentation, Visual Analytics

 

Share |