1.        Introduction

Internet e-marketplaces continue to receive the attention of the business and academic communities as they strive to understand how to use them to leverage the benefits of the digital economy. Many studies deal with the benefits of e-marketplaces and their critical success factors (Bakos 1998; Braun 2002; Brunn, Jensen, & Skovgaard 2002; Le 2002; Raish 2001). However, there have been few attempts to develop evaluation models with a multi-stakeholder perspective. This is because e-marketplace models are evolving and while there are common benefits, some can be differentiated on the basis of e-marketplace structure and ownership models. One factor that predicates the need for an e-marketplace evaluation-framework is the number of government-sponsored regional e-marketplaces (REMs) that are being considered and established (for example the London Marketplace, the Roses Marketplace, Essex marketplace and the Slough IDeA:marketplace in the United Kingdom) despite past failures (Tasmania Business Online e-marketplace (Hayes 2004), Food Connect Australia (Wilkins, Swatman & Castleman 2003) and Bizewest (Western Melbourne Regional Economic Development Organization (WREDO) 2003) in Australia).

The growth of REMs is a result of direct intervention by local governments who view them as a means to promote regional growth and encourage uptake of e-commerce by SMEs. This is one response to research showing that only a small proportion of SMEs were realising substantial benefits from the Internet because of lack of technological expertise and uncertainty about the benefits offered by e-commerce (Daniel, Wilson, & Myers, 2002; Poon & Swatman, 1997; Walczuch, Van Braven, & Lundgren, 2000). However, governments need to balance the desire for regional economic growth via REMs with a proper evaluation of the costs and benefits associated with developing and managing them. Failure to do so could result in wasted public funds and losses to the SMEs involved. Tonkin (2003) likens the use of government procurement REMs to achieve a broad range of policy objectives in the absence of adequate evaluations, as little more than an act of blind faith. Examples of REMs that have ceased operations are Bizewest (Western

While existing IS success models like the Updated DeLone & McLean IS Success Model (DeLone & McLean, 2003) (hereafter referred to as the Updated D&M IS Success Model) can be used to measure the success of e-commerce information systems, specific benefit-evaluation frameworks for REMs can provide existing and potential market makers with a clear idea of the costs and benefits to be considered. This would be particularly useful given the complexities involved in establishing and maintaining REMs and as research indicates that generally the investments needed to create e-marketplaces are very high (Brunn et al. 2002).

This paper uses the REM Success Model (Gengatharen & Standing, 2003a) to evaluate a REM in Western Australia. The strengths and limitations of the model are discussed in relation to how it can be used to help market makers and participants recognise the costs and benefits associated with REMs.

2.        REM success model

Gengatharen & Standing (2003a) propose a conceptual model to evaluate government-sponsored REMs (figure 1). The model, an extension of the Updated D&M IS Success Model, takes a longitudinal approach and considers the context of the evaluation. This includes analysing the motives for the development of the REM using a stakeholder perspective and assessing the profile of the regional SME sector.

Figure 1: REM Success Model (Gengatharen & Standing 2003a)

In figure 1, the context of the evaluation is illustrated in two areas. Firstly, the benefits accruing from the REM depend on the structure of the marketplace and its ownership model, which in turn are dependent upon the market maker’s motivation for REM development and regional SME profile (see Gengatharen & Standing (2003b) for a discussion on market-makers’ motives). Secondly, evaluation of benefits depends on which stakeholder’s view is being considered i.e. buyers, sellers, owners, intermediaries, other stakeholders or the region. Quality of the content, system and value-added services of the REM determine the participants’ intention to use, their actual use and satisfaction with the REM. The more satisfied they are with the REM, the more participants will use it and this determines the benefits that they obtain from using it. The benefits then reinforce the participants’ intention to use, their actual use and satisfaction with the REM. The longitudinal aspect of the REM Success Model considers the evolving nature of REMs in order to identify the benefits that may be experienced further along the REM maturity curve.

2.1         Costs

By including market maker’s motivation and SME profile within the evaluation model, the market maker is provided with an idea of the costs associated with addressing these issues in REM development. For example, the network motive will require the market maker to consider the cost of creating liquidity on the REM. This could involve low participation fees and low cost training or advice to SMEs on e-commerce and REM participation. Grewal, Comer & Mehta (2001) hypothesised that organizational motivation and ability are important determinants of e-marketplace participation. There is a need to identify key players, determine REM features that would motivate them to join and help them to migrate their transactions to the e-marketplace to create early liquidity (Grewal et al., 2001). This has implications for the level of funding needed to develop and maintain the REM. The economic motive would involve reducing costs for participants and improving efficiency possibly through integration of the REM system with that of participants. Table 1 summarises the costs that could be associated with a REM. It has been derived from the following sources: the extant literature on e-marketplaces, IT/e-Commerce evaluation and SME uptake of e-Commerce and our on-going involvement with a REM in Western Australia.

Table 1: Summarised Costs of REMs

2.2         Benefits

To determine the benefits of a government-sponsored REM, Gengatharen & Standing expanded the success metrics from the Updated D&M IS Success model to include benefits identified in the literature on e-commerce (Molla & Licker, 2001; Zhuang & Lederer, 2003, p. 71) and e-marketplaces (Bakos, 1991; Benjamin & Wigand, 1995; Kaplan & Sawhney, 2000; Standing & Stockdale, 2001). The context of the evaluation was also considered by expanding the net benefits category according to individual/organizational/ industry benefits and regional benefits. Table 2 summarises the REM success metrics.

Table 2: REM success metrics

3.        Research design

The previous sections of the paper have highlighted the complexity of evaluating the success of REMs. We want to determine if the constructs identified in the REM Success Model (figure 1) provide a comprehensive framework for identifying factors related to: The motivation of the owner/sponsor, the SME profile, and REM measures of success (including costs and benefits). Therefore, our major research question is:

“Is the REM Success Model a comprehensive framework for evaluating the success of a REM?”

To answer this question and evaluate the REM Success Model we have used a case study approach. Case studies can be used to evaluate theory (Yin 1994, p.38) to determine if the constructs are valid. They are useful when a phenomenon is broad and complex and when an holistic in-depth investigation is needed (Dube & Pare 2003).

For the case study, background data on the REM was collected from official documents, through face-to-face discussions, e-mail correspondence and attendance at meetings with the REM owners. To date, in-depth semi-structured interviews of between 1 to 1½ hours duration have been conducted with nine owner representatives and seven SME participants as the REM has only been actively promoted since the beginning of 2004. The owner representatives were the REM project manager, the economic development managers and heads of purchasing of the towns and REM board members representing the towns and business associations. The interview schedules were designed around the success metrics in the REM Success Model. For the SMEs, a preliminary questionnaire relating to company demographics (e.g. nature of business, size of company, date of incorporation, number of employees, annual turnover, etc.) and REM usage (e.g. date of joining, costs of participation, levels of IT competence, REM usage, monthly business volume via the REM etc.) was distributed and collected back before the interviews. Responses were discussed with participants during the interview. The interview also consisted of questions relating to the participants’ perception of the REM content, system and service quality, as well as actual or perceived benefits of their participation. For owner representative interviews, statistics on REM usage, set-up and ongoing costs, REM income according to type etc. were collected prior to the interviews. The data was discussed during the interviews together with questions about perceptions of the REM and its offerings and actual and perceptual benefits to themselves, SMEs and the region.

The data collected during the study was transcribed and stored in a qualitative research software package. The constructs from the REM Success Model were used as tags and a structural framework developed. Findings indicate that there appears to have been a lack of understanding of the actual costs and benefits of the REM by owners and participants. The use of the REM Success Model and its success metrics has helped owners and participants obtain a clearer idea of these costs and benefits. Data gathering from interviews will continue and a second round of data collection is planned in six months to a year to cater for the longitudinal aspect of the model. The REM Success Model will also be used in cross case analysis.

4.        Case study

TwinTowns (a pseudonym) is a web portal incorporating community content, a business directory and a REM set up by two neighbouring towns in Western Australia. For the purposes of funding and bargaining power, the owner/sponsor group of the portal was widened to include the local councils and business associations of the two towns, and two local higher education institutes (HEIs). The REM is intended to be an electronic gateway to access and interact with local players in the area. Local registered suppliers pay an annual fee of A$ 199 and an additional fee of A$ 99 for a business flyer page. Community groups can have content hosted free of charge on the portal.

The idea of the portal was first conceived in 1999 and with a small seed-funding grant from the state government, a demonstration site was created and used to promote the project and obtain further funding. This was obtained from federal and state government sources and the two towns, their business associations and one of the HEIs. It was intended that after the initial funding was exhausted, TwinTowns would be self-sustainable using income from participation fees, advertising fees and sponsorship. Development of the portal was outsourced in early 2002 and in December 2002, the portal went live with a ‘soft-launch’.

The presence of the REM within the portal is intended to provide SMEs in the region with a low cost, low technology-compliance introduction to e-marketplace trading. The business directory in the REM will allow consumers to source products from local suppliers. The REM also provides an e-procurement mechanism for local and other buyers to request for and receive quotations from registered local SME suppliers and to place orders with them. The key motivations for developing the REM are increasing e-commerce adoption by SMEs, improving business efficiency and increasing trade within the locality, expansion into new markets and development of the region generally.

To date, TwinTowns has a total of 157registered REM participants and 37 community groups while close to A$ 400,000 has been expended on the project. A shortfall in expected funding from government sources, a huge underestimation of the resources required and problems with the software vendor have seen the motive of TwinTowns change to pure survival. Only since the beginning of 2004, with sponsorship funds from a real estate developer, a sales and marketing plan has been put in place. This has seen the number of participants increase by 50 percent. The REM is now being sold on the basis that only discretionary purchasing of the major regional buyers will be channelled via the REM. This is because the e-procurement systems of the major regional buyers cannot be integrated with the REM system.

5.        Applying the REM success model

The following is a discussion of the application of the REM Success Model to TwinTowns. Although only 7 registered SMEs have been interviewed so far, it is nevertheless a good representation as requests for quotes (RFQs) have only been issued since early 2004.

5.1         Market- Maker’s motivation

In TwinTowns, the motivation to stimulate regional development by encouraging SMEs to participate in the REM via a ‘buy local’ policy resulted in the horizontal nature of the market and the RFQ/Ordering mechanism. Entry and technology-compliance costs for SMEs were kept low but lack of resources meant that the portal is currently trying to survive by promoting the REM, while the community side of the portal lies dormant. At this stage, only the discretionary purchasing of the two towns is beginning to be routed through the REM. It is interesting to note that while the majority of the owner representatives interviewed stated that the original motive of the REM was to “promote regional economic growth” by using the REM to stimulate a “buy local policy” and create “local opportunities”, no benchmark figures were available to determine how much spending by the two towns was being channelled into the local market in the first place. Those figures are now being compiled. Another interesting point is that although economic development was claimed as the motive, the original business plan was based on the REM being self-sustainable within two years of operation. This was attributed to the “hubris surrounding the dot.com hype” and “an incrementalist fund-sourcing strategy” where additional funding will be sought after results can be demonstrated.

5.2         Regional SME-profile

SME-profile will influence the structure of the REM and inter-alia the type, level and timing of benefits that can be delivered. While TwinTowns was envisaged initially as a B2B, B2C and B2G REM, local suppliers comprised mainly micro businesses with fewer that 10 employees. All seven SMEs interviewed were in this category and most did not consider using the REM for purchases. One saw it as an avenue for B2C commerce given that the community part of the portal was in the initial plan. Of the remaining 6 SMEs, 5 saw the REM as an opportunity to get a slice of local government business and “hopefully some from the other major local buyers”. However, there was no attempt by the proponents of the REM to determine how the REM system could be integrated with the e-procurement systems of the local major buyers, although some of the owner representatives indicated that the IT department of one of the towns did try to voice their concerns.

A region having strong offline SME networks or alliances may provide impetus for early on-line collaboration in REMs. In TwinTowns, the local business associations represent less than twenty percent of the businesses in the region and collaborative-commerce is not something the SMEs are familiar with. Although complaining about the local governments’ bulk discount purchases from large non-local companies, when the question of collaboration and aggregation of supplies by small local suppliers was raised, one SME replied that it was “pie in the sky stuff” and “if it was me & I had to deal with 5 or 6 other people to get a contract, the hassle that goes with it will be too great”.

Regional SME attitude towards growth will also have a bearing on the structure of the market, its features offered and the benefits to be gained. If the majority of SMEs are lifestyle SMEs (Jeffcoate, Chappell, & Feindt 2000), the owners of the REM may choose to consider an e-marketplace model that reflects the SME owners’ strategy for business growth (Levy & Powell 2003) and market the REM as such.

5.3         Content quality

A number of studies indicate that content quality can affect the satisfaction of web site users (Molla & Licker 2001, p. 138). In the case of TwinTowns, there has been little maintenance of the database of registered suppliers. For example, when the authors tried to send an email to a REM participant, the email was no longer valid. Of the seven SMEs interviewed, four indicated they would not use the REM to look for accurate details of local suppliers, while the others could not comment as they only went into the REM to look at their own listings “some time ago”. On the question of relevance of the portal content, 3 SMEs said they had “no idea” but one qualified the statement by saying that “at that time (more than a year ago) it seemed OK”. Of the remaining 4 SMEs interviewed, one thought the information was relevant (this SME has obtained an RFQ via the REM and has become a supplier to one of the major local buyers), two felt that there was insufficient content and the fourth, a newly incorporated IT services business, felt that the content was ”not useful” and appeared “unprofessional”. Although all the SMEs felt that privacy, security and authenticity of the content were important none felt that these aspects had been tested given the very low level of activity on the REM.

5.4         REM system quality

For many of the SMEs registered with TwinTowns this is their first encounter with e-marketplace participation and system performance is critical to their satisfaction and continued use. The majority of the SMEs interviewed felt that the REM system was either unreliable or they were unable to comment as they had not used it for some time. However they felt that reliability of the REM system would be vital if they were actively transacting through it.

The extent to which the REM system can be integrated with participants’ systems is also a measure of the system’s quality. This is where the longitudinal approach to benefits evaluation comes into play. In the short term the need for integration with the SMEs’ systems may not be critical; it could assume a bigger role as SMEs move further along the e-commerce adoption ladder towards the theoretical end-point of becoming e-businesses (Commission of the European Communities, 2003; DTI, 2001). The majority of SMEs interviewed have not given much thought to integration of the REM system with theirs, but agreed that it could be useful. The longitudinal evaluation is also important as the REM may evolve and its motives could change. Although the success of TwinTowns now appears to hinge on B2G commerce, there appears to be no consideration by the towns of their costs and benefits of procuring through the REM. For example, the REM system is not integrated with the e-procurement systems of the towns and they are running two separate systems for their procurement needs, which could impact on efficiency. Although the REM will be used for the discretionary purchasing of the towns (currently each prospective supplier is manually contacted for quotes), any efficiency savings to them have not been factored into the return on investment of the REM, despite claiming that one of the benefits was the “time and efficiency savings in the process of discretionary purchasing”. In terms of ease of use, all SMEs interviewed rated the REM positively and felt it would be a convenient tool, if there was critical mass.

5.5         Service/value added quality

The service that the REM performs in providing SMEs with e-business training and REM usage, in connecting them with trusted providers of intermediary services (Lenz, Zimmerman, & Heitman 2002) and in offering adequate technical help will have a bearing on the REM participants’ satisfaction and use of the REM. With the TwinTowns REM, although there were plans to provide bundled services to participants by tying up with service providers in the region, these have now been abandoned as the REM struggles to survive. The issue of trust in a government-sponsored REM is one worth mentioning. In TwinTowns, the initial move to register buyers on the REM provided some mechanism of authenticity or trust. However this has now been discontinued, as it is too costly. Feedback can also affect the success of a REM. In TwinTowns, there has been no feedback from the system to participants who have not been sent RFQs, to the extent that the majority of them doubted that the system was working. There are plans now to send out fortnightly or monthly listings of RFQs issued via the REM and reports on REM success stories. Some of the SMEs indicated that knowing who the tender was awarded to could also allow them the opportunity of approaching the awardee for sub-contract work, while a number of the SMEs felt that a feedback system would provide some transparency of the tendering process.

5.6         Use

The use of the REM covers activities like obtaining information (measurable by the number of hits), performing transactions (measurable by number of transactions) and participating in networks (measurable by the presence of and participation in clusters and forums). While there has been some argument over ‘use’ as a measurement of success in IS evaluation, use of a government-sponsored REM is at the discretion of the participant and is therefore a good indicator of REM success. Only 3 of the seven SMEs interviewed have continued to use the REM as they have received RFQs via the system. However, they only use it for responding to RFQs and not for other purposes.

5.7         REM participant satisfaction

This is a measure of how participants feel about all aspects of the REM. Feedback mechanisms can be used to measure satisfaction, as can indicators like repeat visits and transactions and discussion forums. While all the SMEs felt that the concept was good, they were not satisfied due to the low volume of business, the lack of promotion of the REM to the wider public and the lack of feedback.

5.8         Net benefits

This is the ultimate measure of REM success, as it will determine the benefits and negative effects that the REM will have for all stakeholders. The longitudinal approach to evaluation is important as some of the benefits may only be realised in the longer term. Some of the REM net benefits measurements would still be the ones “developed and tested for IS investments in general” (DeLone & McLean 2003, p. 25). These would measure the benefits that fall under the individual and organizational net transactional benefits categories. While the majority of SMEs have not seen any benefits from their participation in TwinTowns yet, they expect that at the very least, if the REM worked as envisaged, they would save on advertising and communication costs. In terms of net strategic benefits, SMEs felt that the REM offered networking opportunities and entry into the “loop of the local major players”. Many of the economic and strategic benefits of the REM for the region will only be realised in the longer term. However the REM Success Model is still useful as it provides an idea of the data that needs to be collected now as benchmarks for future measurement and evaluation. In terms of collaboration and partnerships, for a start, some of the SMEs and owner representatives felt that the project has brought the major stakeholders in the region together in a collaborative effort to improve e-commerce uptake in the region. Not surprisingly all sixteen interviewees felt that the REM was not successful because of low usage resulting from lack of promotion, funding shortfalls and technical problems.

6.        Conclusion

The number of REMs being developed for SMEs, often where the market makers and/or participants do not have a full understanding of the costs and benefits associated with them, predicates the need for an evaluation framework that can encompass a more holistic approach to e-marketplace evaluation. The REM Success Model is a useful evaluation framework that can be used to design instruments to measure the costs and benefits associated with REMs. By using the extant literature on e-marketplaces, IT/e-Commerce evaluation and data from an actual REM, a list of costs was determined according to each type of stakeholder in the REM. Given the wider socio-economic objectives of REMs, the success metrics identified in the framework were not only direct benefits, but indirect ones as well. According to Bakos (1991), e-marketplaces are socially desirable when net welfare gains are greater than development and operating costs. By identifying the costs and benefits to all stakeholders, the REM Success Model helps determine the baseline data that needs to be collected against which to measure success or lack thereof. The REM Success Model is both descriptive and prescriptive as it illustrates the success factors of the REM while also indicating how they can be measured. Future research will include testing the model on other REMs.

References

• Bakos, Y 'A Strategic Analysis of Electronic Marketplaces' MIS Quarterly, Vol 15 No 3 (1991) pp 295-310.

• Bakos, Y 'The emerging role of electronic marketplaces on the Internet' Communications of the ACM, Vol 41 No 8 (1998) pp 49-55.

• Benjamin, R and Wigand, R 'Electronic markets and virtual value chains on the information superhighway' Sloan Management Review, Vol 36 No 2 (1995) pp 62-72.

• Braun, P 'Digital Knowledge Networks: Linking Communities of Practice with Innovation' Journal of Business Strategies, Vol 19 No 1 (2002) pp 43-54.

• Brunn, P, Jensen, M and Skovgaard, J 'eMarketplaces: Crafting A Winning Strategy' European Management Journal, Vol 20 No 3 (2002) pp 286-298.

• Commission of the European Communities, 2003. Adapting e-business policies in a changing environment: The lessons of the Go Digital initiative and the challenges ahead [Online]. Available from: http://www.eu.nl/comm/enterprise/ict/policy/doc/com_2003_148_en.pdf [Accessed: 21May 2003].

• Daniel, E, Wilson, H and Myers, A 'Adoption of e-commerce by SME's in the UK: towards a stage model' International Small Business Journal, Vol 20 No 3 (2002) pp 253-268.

• DeLone, W H and McLean E R. 'The DeLone and McLean Model of Information Systems Success: A Ten-Year Update' Journal of Management Information Systems, Vol 19 No 4 (2003) pp 9-30.

• DTI. (2001). Business in the Information Age. International Benchmarking Report [Online]. Available from: http://www.ukonlineforbusiness.gov.uk/main/resources/publication-htm/bench2001.htm [Accessed: 7 May 2003].

• Dube, L. & Pare, G. (2003). ‘Rigor in Information Systems Positivist case Research: Current Practices, Trends, and Recommendations’ MISQ, Vol. 27 No 4 (2003) pp. 597-635.

• Gengatharen, D. E., & Standing, C. (2003a). 'A Proposed Model to Evaluate the Benefits of Government-Sponsored Regional Electronic Marketplaces for SMEs: Extending the Updated DeLone & McLean IS Success Model'. In Proceedings of the 4th International We-B Conference (Croft, L., Lachowicz, E. & Knight, S. Eds.) We-B Centre, Edith Cowan University, Perth, Australia.

• Gengatharen, D E and Standing, C (2003b). 'A Conceptual Framework to Support the Development of Government-Sponsored Community-Portal Regional Electronic Marketplaces for SMEs: A Stage Approach' In Proceedings of the 14th Australasian Conference on Information Systems (ACIS) (Lethbridge, N. Ed.), We-B Centre, Edith Cowan University, Perth, Australia.

• Grewal, R, Comer, J M and Mehta, R 'An investigation into the antecedents of organizational participation in business-to-business electronic markets' Journal of Marketing, Vol 65 No 3 (2001) pp 17-33.

• Hayes, S. (2004, June 1). Island’s $1.8 e-mart crashes. australian IT.com au. [Online]. Available from http://australianit.news.com.au/com…/0,7208,9706933%5E16123%5E%5Enbv%5E,00.htm [Accessed: 2 June 2004].

• Jeffcoate, J, Chappell, C and Feindt, S 'Attitudes towards Process Improvement among SMEs Involved in E-commerce' Knowledge and Process Management, Vol 7 No 3 (2000) pp 187-195.

• Kaplan, S and Sawhney M 'E-Hubs: The new B2B Marketplaces' Harvard Business Review, Vol 78 No 3 (2000) pp 97-103.

• Le, T T 'Pathways to Leadership for Business-to-Business Electronic Marketplaces' Electronic Markets, Vol 12 No 2 (2002) pp 112-119.

• Lenz, M, Zimmerman, H-D and Heitman, M. 'Strategic Partnerships and Competitiveness of Business-to-Business E-Marketplaces: Preliminary Evidence from Europe' Electronic Markets, Vol 12 No 2 (2002) pp 100-111.

• Levy, M and Powell, P 'Exploring SME Internet Adoption: Towards a Contingent Model' Electronic Markets, Vol 13 No 2 (2003) pp 173-181.

• Molla, A and Licker, P S 'E-Commerce Systems Success: An Attempt to Extend and Respecify the Delone and Maclean Model of IS Success' Journal of Electronic Commerce Research, Vol 2 No 4 (2001) [Online]. Available from: http://www.csulb.edu/web/journals/jecr/ [Accessed: 21 March 2003].

• Poon, S and Swatman, P M C 'Internet-Based Small Business Communication: Seven Australian Cases' Electronic Markets, Vol 7No2 (1997) pp 15-21.

• Raish, W D The eMarketplace. Strategies for Success in B2B Ecommerce, McGraw-Hill, New York (2001).

• Standing, C and Stockdale R (2001). 'Evaluating the benefits of Electronic Marketplaces' In Proceedings of the 2nd International We-B Conference (pp. 118-127). We-B Centre, School of MIS, Edith Cowan University Perth, Australia.

• Tonkin, C., 2003. eProcurement in the Public Sector: Story, Myth and Legend. Working Paper. In: The Policy Institute, Trinity College, Dublin. [Online]. Available from: http://www.policyinstitute.tcd.ie/working_papers/PIWPO7%20-%20Tonkin.pdf [Accessed: 27 January 2004].

• Western Melbourne Regional Economic Development Organisation (WREDO)., 2003. bizEwest B2B Portal. [Online] Available from: http://www_melbwest.com.au/wredo/projects/bizewest.html [Accessed: 19 January 2004].

• Wilkins, L, Swatman, P M C and Castleman, T, 2003. Electronic Markets and Service Delivery: Governance and Related Competencies in Virtual Environments. In: School Working Papers - Series 2003, SWP 2003/02. School of Information Systems, Deakin University, Melbourne, Australia. [Online]. Available from: http://www.deakin.edu.edu.au/infosys/research/working_paper.htm [Accessed: 13 August, 2003]

• Walczuch, R, Van Braven, G and Lundgren, H. 'Internet Adoption Barriers for Small Firms in The Netherlands' European Management Journal, Vol 18 No 5 (2000) pp 561-572.

• Yin, R. K.). Case Study Research Design and Methods (2nd Ed). Sage Publications, Thousand Oaks (1994).

• Zhuang, Y and Lederer, A L 'An Instrument for Measuring the Business Benefits of E-Commerce Retailing’ International Journal of Electronic Commerce, Vol 7 No 3 (2003) pp 65-99

EJISE is published by Academic Conferences Limited
Curtis Farm, Kidmore End, Nr Reading RG4 9AY, England
Tel: +44 (0)1189 724148, Fax: +44 (0)1189 724691, Email: info@ejise.com

Send mail to info@academic-conferences.com with questions or comments about this web site.
Copyright © 2002-2006 Electronic Journal of Information Systems Evaluation
Last modified: September 29, 2005
ISSN 1566-6379