|
The paper titled "Is a Multi-criteria Evaluation Tool Reserved for Experts?", by Camilius Sanga and Isabella M. Venter, presents research to determine whether the analytical hierarchy process algorithm is suitable for the evaluation of software by evaluators who have little Information Technology experience. The scope of the research was the evaluation of two free and open source e-learning systems (Moodle and ATutor) at a university in Tanzania.
Data analysis was done using a holistic approach consisting of both quantitative and qualitative research methods. The results showed that the evaluators at this university preferred Moodle over ATutor. It was also found that the analytical hierarchy process algorithm is appropriate for the evaluation of software in an environment where Information Technology experience is limited. Furthermore, the paper demonstrates the feasibility of using the analytical hierarchy process algorithm in solving a multi-criteria evaluation problem by a group of evaluators. According to the literature the algorithm has many advantages such as: it is easily understood and flexible, it integrates deductive approaches, it acknowledges interdependence of elements of software systems, it has a hierarchical structure, measures intangibles, tracks logical consistency, gives an overall estimation, considers relative priorities and improves judgment.
The authors conclude the paper by describing some of the limitations of the algorithm. The algorithm has a linear modeling characteristic which causes a non-compensatory effect in that it does not reflect the individual value of criteria but rather an aggregate of it. Another inadequacy of the algorithm is that it can only handle exact or crisp evaluations and cannot handle evaluation judgments that are imprecise, ambiguous, uncertain or fuzzy.
Thus, in a future study, the authors will be applying either the analytical network process algorithm or Fuzzy analytical hierarchy process algorithm to address the limitations of the analytical hierarchy process algorithm.
Keywords:
free and open source software, e-learning systems, software quality, multi-criteria evaluation tool, analytical hierarchy process, novice user, developing country
Download FULL PAPER
Back to Contents |
