Paper 4 - Issue 1

An Exploration of the Use of Grounded Theory as a Research Approach in the Field of IS Evaluation.
Steve Jones, Information Systems Research Centre, University of Salford, Salford, UK. Jim Hughes, School of Accounting and Information Systems, University of South Australia, Adelaide, Australia. steve.jones@conwy.gov.uk ; j.hughes@salford.ac.uk

1. Introduction

There is an enduring debate within the IS field concerned with the positivist and interpretivist research traditions (Fitzgerald and Howcroft, 1998). Undoubtedly, positivism has been most influential, with much of the published research being centred upon positivistic assumptions (Orlikowski and Baroudi, 1991; Walsham, 1995; Galliers, 1997). Indeed, much of the IS literature assumes an unproblematic, rational, objective, approach to management, organisations, and change, involving mechanistic assumptions about organisational behaviour (Introna, 1997). However, a mounting appreciation of the essentially social nature of IS in practice has led some researchers to adopt research approaches which focus primarily on human interpretations and meaning (Walsham, 1995; Myers, 1997). Consequently, interpretive approaches are being employed to an increasing extent (Klein and Myers, 1999).

There is also increasing evidence of this debate in the field of IS evaluation. Many authors (Walsham, 1995a; Hirschheim and Smithson, 1999; Serafeimidis and Smithson, 2000; Irani and Love, 2001) have argued that IS evaluation would be improved by using an interpretive epistemology. The emerging view is that social actors, that is the people involved in the IS evaluation situation, are important and that IS evaluation is a socially embedded process in which formal procedures entwine with the informal assessments by which actors make sense of their situation. This view argues that these actors are in the best position to assess IS, offer opinion, and persuade senior executives of the value and success of the IS. Notable amongst the interpretivist researchers in the field of IS evaluation are Serafeimidis and Smithson (2000). Using an interpretive methodology, they contend that the use of IS is governed by actor attitudes.

For researchers who adopt an interpretive research strategy a more focussed, and possibly problematic, issue is that of selecting appropriate research methods to complement the chosen strategy. Not surprisingly, many interpretivists select qualitative research methods, given the popular misconception that interpretive research can be equated with qualitative research and all the associated procedures for qualitative analysis (Myers, 1997). However, the assumption that qualitative methods are the only ones suitable for interpretive research, or conversely, that quantitative methods should only be used for positivist research, is a fallacy (Klein and Myers, 1999; Nandhakmar and Jones, 1997). There are examples where quantitative methods, such as statistical analyses, are employed in interpretive research (Kaplan and Duchon, 1988), likewise, it is possible to employ qualitative methods, such as case studies, which are positivist (Yin, 1989). Nevertheless, whilst it is incorrect to equate qualitative research with interpretivism, it does tend to be the method most frequently drawn upon.

Therefore, arising from this, an issue of concern for many interpretive researchers, is that when presented with unstructured, non-numeric data derived from a qualitative research study, such as data from interviews, there is a noticeable absence of practical guidance for the coding and analysis of data. An element of relief can be found, however, with Grounded Theory (Glaser and Strauss, 1967), one of the practical methods that is prominent amongst the various research strategies recommended for conducting interpretive research (Myers, 1997).

The paper is structured as follows. In the following section the authors give an overview of GT, its history and procedures. This section is followed with an analysis showing the various underlying assumptions adopted when GT has been used in IS research. This is followed by an illustrative example of the use of GT in IS evaluation research which draws upon the authors’ own work in this area. The paper continues with a discussion of the issues that arise from the use of GT and gives guidance on the adoption of the method and its appropriateness in the IS evaluation field.

2. The Discovery of Grounded Theory

Glaser and Strauss (1967) were the original proponents of the set of procedures or method which enabled locally generated empirical theory to be discovered. The title of their method, which was the title of the original book, The Discovery of Grounded Theory, is widely misunderstood. The method is concerned with discovering theory that is grounded in social settings. The title of the original book reflected this; it is not a book about their discovery of a method that they called Grounded Theory. The misconception is so widespread that the method is now commonly known in its shortened form and indeed following common practice it is the name that the authors will use in this paper.

GT is a method for the collection and analysis of qualitative data developed by Glaser and Strauss (1967). Although the method dates back to 1967, its use in Information Systems research is very recent. It is growing in popularity, particularly in the interpretivist IS research paradigm, for enabling rich and context sensitive analysis of social situations (Baskerville and Pries-Heje; 1999; Urquhart, 1999; Hughes and Wood-Harper, 1999; Trauth, 2000). Useful summaries of the use of GT in IS research can be found in Hughes and Howcroft (2000) and Urquhart (2001).

In the method, conceptual properties and categories may be ‘discovered’ or generated from the qualitative data by following a number of guidelines and procedures where the aim is to indicate the importance of the actors in the social setting. It is they who provide the transcript data from interviews. As soon as the first transcript is prepared, coding begins. The term ‘coding’ is used here to describe the process of annotating or labelling interview transcripts with a piece of text. At this stage the coding is ‘open’ and is fairly unrestricted, in the sense that each line, and sometimes each word, is scrutinised in order to produce a code which initially may appear as a simple repetition or summary of the text. The purpose is to ‘open’ the inquiry into the data and at this stage any element of the data may seem relevant. As more data is collected, the researcher continues the open coding, but also begins to identify ‘categories’ rather than labels. The categories are formed from groups of words from the initial coding which pertain to the same phenomenon. Categories are at a higher level and more abstract than the concepts they represent. The concepts must be repeatedly present in the data to warrant the abstraction to the level of category. The researcher may select both in vivo categories and their own categories. The categories are generated through the same analytic process of making comparisons in order to highlight similarities and differences.

For the process to be rigorous, the researcher must go back through the currently available data and re-code in relation to the new categories. Researchers may then use their knowledge of the domain, or draw upon their previous experience, in order to expand and clarify categories and sensitise themselves to ways of further exploring the data. As categories are formed it is essential that each have an associated memo, which elucidates it and relates it back to the data.

As the number of categories increase they can be related to each other and to sub-categories. This process is known as ‘axial’ coding, in which the researcher explores the relationship amongst categories and sub-categories. This will usually necessitate the gathering of further data to explicate quite specific points in order that a category can be saturated. Saturation of a category occurs when the collection of data and its analysis can add no further to the properties of the category. Categories may also be related to each other to provide an integrating framework. During axial coding the analytical memos will now be developing the properties of categories and the relationship with sub-categories, in particular the integration of categories and the relationship between them.

As the number of saturated categories grow and the relationship between them becomes established they can be unified around a ‘core’ category. This core category represents the central phenomenon of the study and usually evolves towards the end of an investigation. At this stage the researcher may have identified several schemes that can link the data together. It is important that a central category is derived which can be achieved using either abstraction or generalisation. Indeed, the more abstract the core category, the more widely applicable it becomes. The ‘filling in’ of poorly developed categories also takes place at this stage, in order that all categories in the final integrative framework are saturated.

Two analytical techniques, theoretical sampling and constant comparison, are the means by which the GT proceeds. These techniques relate the coding activities to the pool of data and memos that already exist. Theoretical sampling is an inductive technique that is used when the data does not exhaust the theoretical category that is being developed. The researcher must return to the domain until categories have been saturated. That is to say the researcher must sample particular categories to ascertain that its properties cannot be elucidated further. This checks both the scope, as well as the depth of a category. The theoretical sampling in GT is theory-driven. Strauss and Corbin (1998) emphasise that the grounds that determine which events, activities, or people require further data collection are analytic grounds based on the emerging theory. As each new incident or concept emerges, it needs to be compared against existing concepts and categories for both similarities and differences.

The purpose of this constant comparison is threefold. Firstly, to attempt to ensure the researcher’s neutrality by constantly being challenged with new data; secondly, to help achieve greater precision in grouping together like phenomena; and thirdly, to further subdivide concepts which are variations on the first, to ensure consistency in grouping like with like. Strauss and Corbin (1998) stress that the researcher should be vigilant in their search for patterns and variations, by arguing that concepts and their dimensions must ‘earn’ their way into the theory. Figure 1 below outlines the GT procedures.

Constant Comparative Analysis

Theoretical Sampling

Figure 1: Grounded Theory Procedures (Hughes and Howcroft, 2000)

3. The Use of Grounded Theory in IS Research

Hughes and Howcroft (2000) consider and analyse the use of GT in interpretive research and highlight a number of examples in the IS literature where the method has been used by IS researchers. In order to clarify this analysis, the characteristics of the projects and the underlying assumptions of the researchers are tabulated in Table 1. The authors accept that the projects do not represent a definitive account of the use of Grounded Theory in IS research, rather they typify the ways in which Grounded Theory has been used in a variety of projects.

The entries in each section are summaries of those that have been reported by the project authors. They are not our interpretation of how Grounded Theory was used - nor would we claim authority. However, we would point out that when considering the use of Grounded Theory in the IS field there are inconsistencies in both the understanding of the method and the application of the method. One could argue that this is because Grounded Theory is more or less appropriate for addressing certain kinds of research questions or at certain stages of the research process. However, whilst further investigation into the relationship between the ways in which Grounded Theory is adapted and the kinds of research questions studied may prove fruitful, we would argue that since Grounded Theory has had limited application, researchers would be well advised to explore its use in different contexts. Therefore, we prefer to avoid making sweeping generalisations about its recommended usage.

The projects range from those concerned with organisational change (Orlikowski, 1993; Pries-Heje, 1992) to those concerned with the practical use of the method to inform knowledge based systems design (Oliphant and Blockley, 1991; Galal and McDonnell, 1997). For some the full-blown use of the method is important (Pidgeon et al, 1991) whilst for others the use of specific procedures helps to overcome deficiencies in another research strategy (Baskerville and Pries-Heje, 1995; Baskerville and Pries-Heje, 1999; Hughes, 1998). Some researchers selectively change the method to suit the purpose of the research (Calloway and Ariav, 1991; Fitzgerald, 1997).

The underlying assumptions made explicit by the authors range from qualitative-interpretive (Howcroft, 1998) to qualitative-positivist (Pries-Heje, 1992). The latter to the extent that the project’s author explicitly commends the method for its meeting the criteria of good science. Although interestingly, the criteria are not rigorously pursued. Grounded Theory is used as a method alongside others (Orlikowski, 1993; and Torsaskar, 1991) and also used entirely on its own (Calloway and Ariav, 1991). The projects are summarised in Table 1 below.

Researcher	Research Project	Underlying Assumptions and Perspectives	Adaptations to Grounded Theory Method Made Explicit
Toraskar	To evaluate the product of IS design.	Grounded Theory is in the tradition of hermeneutic methods of analysis and prediction.	Ignores evaluative criteria.
Calloway and Ariav	To explore how IS designers perceive the use of design tools during systems development.	Grounded Theory can be used as a method of data reduction alongside Content Analysis.	Ignores evaluative criteria. Uses seed categories rather than inductive generation of all categories.
Pidgeon, Turner and Blockley	For conceptual analysis in knowledge elicitation.	Grounded Theory provides rich conceptual models that accurately describe data. The evaluative criteria provide necessary rigour.	Follows the method.
Oliphant and Blockley	Design of a knowledge based advisor on the selection of earth retaining structures.	Grounded Theory provides a hierarchical model from which knowledge based rules can be derived.	Follows the method.
Pries-Heje	An investigation into barriers for the use of computer-based tools in ISD.	Grounded Theory provides systematic techniques to develop substantive theory that meet the criteria of ‘good’ science	Follows the method
Orlikowski	Development of a conceptual framework for understanding organisational issues surrounding the adoption and use of CASE tools.	Grounded Theory adopted because of its inductive, contextual and procedural characteristics useful in considering change	Follows the method alongside other methods to triangulate findings
Baskerville and Pries-Heje	To study methods and tools in order to improve problems that exist in software product development.	Grounded Theory increases the rigour in the theory development portion of action research.	Limited theoretical sampling. Predetermined core categories. Reorientation of early coding to evaluate and learn about predetermined core category.
Fitzgerald	A field study of the use of systems development methodologies in practice.	Grounded Theory increases rigour and traceability during theory development.	Use of seed categories.
Galal and McDonnell	A case study of an automated document management system for the legal domain.	Grounded Theory supports rigorous analysis of qualitative data. Adopted for the evolution of first requirements’ model.	Follows the method integrated into KBS engineering techniques.
Urquhart	A case study of analyst-client communication during early requirements gathering.	Grounded Theory offers well-signposted procedures which aim to produce a theory that is precise, rigorous and capable of replication.	Follows the method alongside other qualitative techniques and questionnaires in order triangulate the findings.
Hughes	A data analysis method used for requirements determination.	Grounded Theory provides useful procedures to elicit situated requirements within an ethnomethodological perspective.	Use of seed categories. Use of software package to aid analysis. Ignores evaluation criteria.
Howcroft	An interpretive study of the nature and characteristics of Internet usage.	Grounded Theory offers a systematic and traceable method for inducing patterns from the raw data.	Use of seed categories.

Table 1: Use of Grounded Theory in IS Researc

This raises a number of questions for the use of Grounded Theory in IS research. Is there some ‘correct’ way of using Grounded Theory? Is there a research paradigm in which Grounded Theory ‘fits’? What validity can be given to the concerns of Strauss and Corbin (1998) about the need for sensitised, trained and experienced Grounded Theorists? The questions are returned to in the Discussion section of the paper. In the following section by way of illustrating of the use of GT as a research method in IS evaluation an interpretive case study is described and the outcomes from the study presented. A full discussion of the case study is given in Jones and Hughes (2000).

4. Illustrative Case Study

The case study in this section concerns one Local Authority in the UK, which provides a full range of public sector services, for example, Education, Social Services, Highways, Housing, Planning and Public Protection. Qualitative methods of data collection, particularly informal semi-structured interviews with senior IS stakeholders, were used together with GT to analyse the data. The use of Grounded Theory identified a core category, that of Information Systems Evaluation, together with three emergent themes, or major categories (figure 2). The major categories are firstly, that IS investment decisions are usually undertaken in isolation by Information Systems Department Management and that they are largely intuitive and political in nature. Secondly, that evaluation methods, traditional or otherwise, are not utilised. Thirdly, that an interpretive approach to IS evaluation would be worthy of consideration.

Information Systems Evaluation

IS Investment Decisions IS Evaluation Methods Situated Practice

IS investment decisions are usually undertaken in isolation by the information systems department management. They are largely intuitive and political in nature

Evaluation methods, traditional or otherwise, are not utilised

An interpretive approach to IS evaluation would be worthy of consideration

Figure 2: Core Category and Major Categories, with brief descriptions

4.1 IS Investment Decisions

The organisation addresses IS investment and evaluation issues via a Corporate Information Technology Client Group (CITCG), regular IS Account Management meetings between the IS Department and IS Users, and monthly IS Departmental Management Team meetings. Arising from these forums the IT Departmental Management Team, consisting of three senior IT managers and the CIO, make recommendations to the CITCG, which are invariably accepted.

The decision to invest in IS therefore, appears at first sight to be strategic, since the CITCG, endorse the choice suggested by the IT Department. There is also a seemingly well-planned and participative approach to IS investment decisions. However, in practice, IS implementations are not introduced through a positive commitment by the Authority in its entirety, as the Director's Management Team and many key stakeholders are not consulted. The decision making process is illustrated in figure 3 below.

Corporate IS Client Group

Ill-informed

Reluctant to be Involved

Uneasy with IS Decision Making

IS Management Meetings Account Management Meetings

IS Perspective Increasing user IS Awareness

Make and Defend IS Decisions Anxious to use IS

Lacks a Strategic View Piecemeal IS Adoption

Figure 3: Decision-Making Process

It is difficult engaging senior managers in the IS decision making process and therefore it is perhaps not surprising that this approach has been adopted. IS investment decisions are made with the intention of fully supporting service delivery plans. However, the prevailing IS decision-making process makes it extremely difficult to determine to what extent this is successful. IS investment is largely intuitive and highly political in nature. The power of politics and persuasion is a major factor when convincing, or otherwise, the CIO when determining IS investment decisions.

4.2 Evaluation Methods

Traditional mechanistic evaluation processes are not utilised, although there is a high level of awareness of them. There is no significant pre or post implementation evaluation of IS value and benefit via any formal review processes, except for an estimate of costs. Decision-makers have a high expectation of IS implementations and assume that IS will always bring benefits. No risk assessment is undertaken and neither are potential dis-benefits identified.

Discussion and discourse during Account Management meetings is widely used to determine the level of satisfaction with IS within the organisation. Consequently, difficulties are raised and, where necessary, problems are formally documented. Questionnaires are also used to ascertain user satisfaction. This is a form of qualitative, interpretive evaluation of IS implementation.

IS practitioners and stakeholders regard formal evaluation methods as being of little value. This is the main reason cited why they have been disregarded. Despite the lack of evaluation methods, it was interesting to note that the £2.5m annual revenue budget was regarded as being spent appropriately by almost all interviewees and that no wastage had occurred, even though it was impossible to demonstrate that this was the indeed the case.

4.3 Interpretive Approach to IS Evaluation

It was acknowledged that IS monitoring, review and evaluation processes are important, to enable problems and issues to be raised, documented and resolved. Despite this acknowledgement, this has not been given high level of importance in the organisation. User opinion is regarded as an extremely important aspect, which is overlooked. This is unfortunate, as it was agreed that actor opinion has the potential to produce deep insights into IS evaluation and could greatly assist in this area. However, what is problematic is the articulation and documentation of the process. This is the main barrier to undertaking interpretive studies in the case study organisation.

The organisation acknowledges that improving IS evaluation may lie, not with the development of better quantitative methods, but rather in the experiences of multiple IS stakeholders. Furthermore, it is aware that practical elaboration of the evaluation is predominantly grounded in tacit knowledge. User opinion was seen as a major factor to consider. Interviewees stated that there are inherent intuitive and political dimensions to IS evaluation.

4.4 Lessons for IS Practitioners and Organisations

The critical aspect of the case study given in this paper is that in its methodology, Grounded Theory, it explicitly sought to discover the underlying assumptions, the contexts and the experiences of those involved in the IS evaluation process. The outcomes from the case study are best expressed as lessons and practical guidelines since no claim is made for interpretive studies of this type for generalisation (Walsham, 1999). However, the richness of the case study does lead to lessons for IS practitioners.

Lesson 1 Organisations must question the underlying assumptions of existing IS evaluation methods and seek alternative perspectives where these methods fail to achieve desired objectives and outcomes.

Lesson 2 IS evaluation must have an explicit concern for the social context and interpretive methods must be adopted which explicitly situate stakeholders at the centre of the evaluation process.

Lesson 3 The views, beliefs and assumptions of stakeholders must be exposed and considered within the IS evaluation process and not be ephemeral to it.

Lesson 4 Organisations must have an interpretive benefits management approach to complement the IS development, project management and implementation functions.

Lesson 5 A new role must be created for an IS evaluation professional to facilitate and conduct interpretive IS evaluation studies.

Lesson 6 The hierarchical and political nature of public sector organisations creates a barrier to change and this must be overcome to ensure interpretive approaches are successful in practice.

Furthermore, the case study points to practical guidelines for organisations with regard to IS evaluation and these are given in table 2 below.

Situated Practice Characteristics	Practitioner Action
Concern for evaluation in practice Interpretive approach	Articulate the importance of the stakeholder view by appointing a facilitator to elicit the views and concerns of stakeholders so that these can be disseminated. Use methods in practice similar to the GT method which provides a set of procedures for the articulation analysis and dissemination of a grounded view of stakeholders
Covert non-documented process	Expose and document these grounded processes
Social factors dominate Promotes subjective views Engages in process Stakeholder view of process Recognises political process	Through seminars and group discussion expose the underlying assumptions and values. This can be done by utilising the hermeneutic approach to give precedence to ‘evaluation in practice’ During this exercise explicitly address the political situation to identify power relations

Table 2: Framework for Understanding Situated IS Evaluation

5. Discussion

The use of GT in the above case study indicates that it can assist conceptually in the understanding of a problem situation and can also assist with articulation of lessons learned. However to return to the questions posed earlier in the paper about the ‘correct’ way of using GT it may be argued by some that the method has been used in a contingent way which does not reflect the procedures or philosophy of the original proponents. However we would argue that from its roots in the social sciences, its philosophy is post-modernist, and that it relies less on dogma and positivist validity models and more on the practical application of methods to suit real situations. In a given methodological context therefore, one would expect that the intended method could differ from the method in use because of the dynamics of the situation and that this is entirely consistent with the need to extract richness from social situation. An important aspect arising from the use of GT in the case study is that agency, that is the attention to the means by which actors or organisational members create and recreate organisational structures, is paramount. Its use implies an organisational context rather than the perhaps more familiar technical or economic contexts for research in the IS evaluation field.

In spite of this rationale for the use of GT in interpretive case studies the above example did highlight a dilemma between the rationale and the GT procedures themselves - coding, comparing, categorising, saturating – all of which have a positivist feel to them. In resolving this one can look to the experiences of other researchers (Klein and Myers, 1999) who claim that the assumption that qualitative methods are the only ones suitable for interpretive research, or conversely, that quantitative methods should only be used for positivist research, is a fallacy. Certainly in the case study above a good deal of the data structuring can be attributed to the researcher (as is common in interpretive research) and the GT methods have their place as framework or guidelines or pointers through this structuring process.

As with any research, every researcher is initially a novice in the use of research methods when applying them in a new environment, especially within a new discipline. A researcher’s early understanding of GT methods can therefore be insufficient and the use of methods without a full understanding of them or of their conjunction can be problematic. This matches the Strauss and Corbin (1998) concern about the need for sensitised, trained and experienced GT researchers. The authors would agree that for the case study above, more extensive practical use of the methods and training in social science research would have been invaluable. However as a discipline ‘we are where we are’ but it serves as a warning to less experienced researchers before borrowing from other disciplines they should have a good prior knowledge of the reference discipline. Based on their own experience the authors tentatively give the following guidance for GT use in IS evaluation research.

· In the methodological context the intended GT method may not turn out to be the same as the method used because of the dynamics in the situation

· GT is consistent with interpretive studies dealing with social and organisational contexts.

· The researcher’s personal constructs and skills help structure data and it is the researcher’s perspective that maintains the interpretive style rather than GT.

· GT can be too time consuming, particularly in the transcribing, coding and comparing associated with the data analysis

· To fully understand GT, practical use of the methods in social science research is invaluable.

· In organisational settings, social science methods such as GT can be useful in providing deep insights and understanding of social life.

· GT provides a useful template for researchers and can serve as a comfort factor for the stressful and uncertain nature of conducting qualitative research.

6. Conclusion

In this paper the authors have highlighted that GT, a method more commonly associated with the social science perspective, can assist with rich, context based IS research. The empirical work has further demonstrated that it is possible to successfully use GT in qualitative IS evaluation studies where the social aspect is paramount.

The traditional perspective views IS evaluation in a scientific sense and positivist, mechanistic evaluation methods are used. The situated approach views IS evaluation in a socially constructed sense. If the latter view is dominant, it follows that there is need to use social science methods to evaluate IS. There is a problem to overcome however, in that although social science methods such as GT may be appropriate, there can be a difficulty in effectively introducing such methods into a technically dominant field such as information systems. The methodology’s acceptability and practicability therefore needs to be more strongly established by further research.

References

Baskerville, R. and Pries-Heje, J. (1995) Grounding the Theory in Action Research, in Proceedings of the Third European Conference on Information Systems, Athens, Greece, 837-848.
Baskerville, R. and Pries-Heje, J. (1999) Grounded Action Research: A Method for Understanding IT in Practice, Accounting, Management and Information Technology, 9(1), 1-23.
Calloway, L. J. and Ariav, G. (1991) Developing and Using a Qualitative Methodology to Study Relationships Among Designers and Tools, in Nissen, H. E., Klein, H., Hirschheim, R. (Eds.) Information Systems Research: Contemporary Approaches and Emergent Traditions, Proceedings of the IFIP WG 8.2 Working Conference, Copenhagen, 14-16 December, Amsterdam, North-Holland, 175-193.
Fitzgerald, B. (1997) The Use of Systems Development Methodologies in Practice: A Field Study, Information Systems Journal, 7(4), 121-132.
Fitzgerald, B. and Howcroft, D. (1998) Towards Dissolution of the IS Research Debate: From Polarization to Polarity, Journal of Information Technology, 13(4), 313-326.
Galal, G. and McDonnell, J. T. (1997) Knowledge-Based Systems in Context: A Methodological Approach to Qualitative Issues, AI & Society, 11, 104-121.
Galliers, R. (1997) Reflections on Information Systems Research: Twelve Points of Debate in Information Systems: An Emerging Discipline? (Mingers, J. and Stowell, F., Eds.), McGraw-Hill, London., 141-154.
Glaser, B. G. and Strauss, A. (1967) The Discovery of Grounded Theory: Strategies for Qualitative Research, Aldine, New York.
Hirschheim, R. A. and Smithson, S. (1999) Evaluation of Information Systems: A Critical Assessment, in Beyond the IT Productivity Paradox (Willcocks, L. and Lester, S., Eds.), Wiley, Chichester.
Howcroft, D. A. (1998) Spanning the Spectrum from Utopia to Dystopia: an Interpretive Field Study of the Nature and Characteristics of Internet Usage, PhD Thesis, UMIST, Manchester, UK.
Hughes, J. (1998) The Development of the GIST (Grounding Information Systems) Methodology: Determining Situated Requirements in Information Systems Analysis. PhD. Thesis, Information Systems Research Centre, University of Salford, Salford, UK.
Hughes, J. and Howcroft, D. A. (2000) Grounded Theory: Never Knowingly Understood, Information Systems Review, 4(1), 181-197.
Hughes, J. and Wood-Harper, A. T. (1999) Systems Development as a Research Act, Journal of Information Technology, 14(1), 83-94.
Introna, L. (1997) Management, Information and Power, Macmillan, London.
Introna, L. D. and Whitley, E. A. (1997) Against Method-ism: Exploring the Limits of Method, Information Technology & People, 10(1), 31-45.
Irani, Z. and Love, P. D. E. (2001) The Propagation of Technology Management Taxonomies for Evaluating Investments in Information Systems, Journal of Management Information Systems, 17(3), 161-177.
Jones, S. and Hughes, J. (2000) Understanding IS Evaluation as a Complex Social Process in Proceedings of the Americas Conference on Information Systems, August 2000, Long Beach, California, USA.
Kaplan, B. and Duchon, D. (1988) Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study, MIS Quarterly, December 1988, 571- 586.
Klein, H. K. and Myers, M. D. (1999) A Set of Principles for Conducting and Evaluating Interpretive Field Studies in Information Systems, MIS Quarterly, 23(1), 67-94.
Myers, M. (1997) Interpretive Research in Information Systems, in Information Systems: An Emerging Discipline? (Mingers, J. and Stowell, F., Eds.), McGraw-Hill, London, 239-266.
Nandhakumar, J. and Jones, M. (1997) Distance and Engagement in IS Research, Information Systems Journal, 7(2), 109-131.
Oliphant, J. and Blockley, D. I. (1991) Knowledge-Based System: Advisor on the Earth Retaining Structures, Computers and Structures, 40(1), 173-183.
Orlikowski, W. J. and Baroudi, J. J. (1991) Studying IT in Organizations: Research Approaches and Assumptions, Information Systems Research, 2(1), 1-28.
Orlikowski, W. J. (1993) Case Tools as Organizational Change: Investigating Incremental and Radical Changes in Systems Development, MIS Quarterly, 17(3), 309-340.
Pidgeon N. F., Turner, B. A. and Blockley, D. I. (1991) The Use of Grounded Theory for Conceptual Analysis in Knowledge Elicitation, International Journal of Man-Machine Studies, 35(2), 151-173.
Pries-Heje, J. (1992) Three Barriers for Continuing use of Computer-Based Tools in Information Systems Development: a Grounded Theory Approach, Scandinavian Journal of Information Systems, 4, 119-136.
Serafeimidis, V. and Smithson, S. (2000) Information Systems Evaluation in Practice: a Case Study of Organisational Change, Journal of Information Technology, 15(2), 93-105.
Strauss, A. and Corbin, J. (1998) Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Sage, London.
Strauss, A. and Corbin, J. (1998) Grounded Theory Methodology: an Overview, in Strategies of Qualitative Inquiry, (Denzin, N. K. and Lincoln Y. S., eds.), Sage, London, 158-183.
Toraskar, K. (1991) How Managerial Users Evaluate Their Decision Support: a Grounded Theory Approach, in Nissen, H. E., Klein, H., and Hirschheim, R. (eds.) Information Systems Research: Contemporary Approaches and Emergent Traditions. Proceedings of the IFIP WG 8.2 Working Conference, Copenhagen, 14-16 December. Amsterdam, North-Holland, 1991, 195-225.
Trauth, E. M. (2000) The Culture of an Information Economy: Influences and Impacts in the Republic of Ireland, Kluwer Academic Publishers, Dordrecht, Netherlands.
Urquhart, C. (1999) Themes in Early Requirements Gathering: The Case of The Analyst, The Client and The Student Assistance Scheme, Information Technology and People, 12(1), 56-71.
Urquhart, C. (2001) An Encounter with Grounded Theory: Tackling the Practical and Philosophical Issues, in Qualitative Research in Information Systems: Issues and Trends (Trauth, E., Eds), Idea Group Publishing, London.
Walsham, G. (1995) Interpretive Case Studies in IS Research: Nature and Method, European Journal of Information Systems, 4(1), 74-81.
Walsham, G. (1995) The Emergence of Interpretivism in IS Research, Information Systems Research, 6(4), 376-394.
Walsham, G. (1999) Interpretive Evaluation Design for Information Systems, in Beyond the IT Productivity Paradox (Willcocks, L. and Lester, S., Eds.), Wiley, Chichester.
Yin, R. K. (1989) Case Study Research: Design and Methods, Sage, London.