1.        Internet banking in Brazil

Internet banking has been reported as the most important innovation for banking services deployment of the last years. In Brazil, like other countries, Internet banking has grown very fast. Since the early adopters in 1996, banks offering Internet banking services jumped from only 9 banks in 1997 to 75 banks in 2003 (Diniz 2004). These numbers are even more impressive if one considers that these banks represented 83% of the total assets of the whole banking system of the country at that time.

In terms of transactions, Internet banking has kept the highest growth rate among all of the banking channels. Between 1998 and 2002, while the whole banking system had increased 20% a year in the total number of transactions, the ones made through the Internet had a growth rate of 180% a year. The figures (CIAB 2003) also show that Internet banking transactions represented more than 10% of all banking transactions in Brazil, in 2002. In order to make a comparison, ATMs, the most used channel for transactions, held at the time something around 30% of the total number of transactions.

Also the number of Internet banking users in the country has skyrocketed. According with studies carried out by the bankers federation (CIAB 2003), the number of clients banking on the Web almost doubled between 2000 and 2003, from around 8 millions to 15 million, less than 10% of them being from company accounts.

In 2003, most of the banking services offered in regular branches and ATMs were also found on the Internet. As shown at the table 1, this phenomenon of large adoption of Internet banking can be observed by all the main banks operating in Brazil. It is also important to notice that the use of Internet banking use in Brazilian banks reaches higher rates than other countries. While, in Europe, Internet banking use rate is around 18% of the clients, in Brazil, similarly with USA, the same rate is around to 23% (Hessel 2003). However, as one can see in the table, some of the main banks in Brazil have surpassed this percentage.

Table 1: Percentage of clients per bank using Internet banking in Brasil

Source: Business Standard*; Gazeta Mercantil**

Although the numbers mentioned above show the growing importance of Internet banking, they do not say anything about how the services are offered. That is exactly the point this article intends to focus on. With many banks offering different on-line services to such a large amount of clients, the question is to develop ways to evaluate the quality of these services. In the next session it will be presented a discussion on Web site evaluation, aiming the development of a framework for evaluate the quality of Internet banking services.

2.        Web site evaluation

Website quality has been evaluated from many different approaches. A number of models derived from the Technology Acceptance Model – TAM (Davis 1989) are used to create a Web evaluation framework (Heidjen 2001; Horton et al. 2001; Lederer and Maupin 2000; Moon and Kim 2001; and others). These models highlight website usability and utility. The WebQual (Barnes and Vidgen 2000; Loiacono et al. 2000) and the e-ServQual (Zeithalm et al. 2002) models add additional analysis dimensions, such as reliability, security and privacy (among others).

In addition to these customer-focused models, the quality of the Web sites can be evaluated from the software development side. Pressman (1997) advances the FURPS – functionality, usability, reliability, performance and supportability – quality factors for software development, and Mendoza et al. (2002) create an ISO/IEC 9126-based portal environments evaluation model.

Certain Brazilian studies on Web evaluation have concentrated on set of on-line services offered (Soares and Hoppen 1998; Diniz 1999; Angulo and Albertin 2000; Nogueira et al. 2000; Steil et al. 2001; Cunha and Reinhard 2001; Mariano et al. 2002). Others studies address browsing- and design-related aspects (Ramos and Costa 1999; Nogueira et al. 2000; Steil et al. 2001; Cunha and Reinhard 2001; Sartori et al. 2002; Oliveira 2002; Ferreira and Leite 2002). A few also investigate transaction security and reliability aspects (Soares and Hoppen 1998; Sartori et al. 2002). A significant share of these works also try to establish criteria and attributes that may be used to evaluate and compare virtual business environments (Cano et Becker 1999; Freitas et al. 2001; as well as others in previous references). 

Considering all these models and studies, as well as the authors’ experience with the evaluation of business websites, three concepts appear common:

1.   Functionality: defines the set of services offered, focusing on business opportunity and strategy;

2.   Reliability: defines the level of transaction security, focusing on the elements that may contribute to user trust;

3.   Usability: defines user interaction with the website, focusing on the ease of browsing and performance of tasks leading to completion of the transaction.

The purpose of this article is to propose and test a model based on these three dimensions and capable of making a contribution to the development of methodologies to evaluate and compare virtual business environments from the perspective of website users.

For each one of the above dimensions, this article presents a coherent set of criteria that may be useful to virtual business environment evaluation and design. We propose an analytical framework for business environments in the Web based on: (i) the set of services offered; (ii) investigation of security requirements adopted in business Websites; and (iii) Website usability.

Figure 1: Evaluation model of digital business environment

In the proposed model, each of these three dimensions is further subdivided into three other levels (Figure 1), according to the level of difficulty involved in implementing the relevant resources at the corresponding dimensions. The outermost layer contains factors more easily implemented on a Web-based business site, the intermediate layer contains medium-difficulty factors, and the innermost layer contains factors most difficult to implement. Below, we discuss each of these factors in greater depth and consider each dimension’s characteristics.

3.        The functionality dimension

Functionality, meaning the online services offered, is the first dimension to consider when studying virtual business environments. Prior to anything else, when a company decides to offer its services on the Web, it must consider which services may best result benefits for the company and its customers and partners. This dimension varies according to the industry and company strategy, as well as the profile of the customers for whom a particular set of services is intended.

Diniz et al.(2002) categorize Web services based on direction of the information flow between users and the corporation over the Web. The authors identify certain technical and organizational requirements for on-line services implementation. One major element of this categorization is its focus on the services offered, which divide into the three categories below:

§         Dissemination: as a vehicle to publish information

§         Transaction: as a channel to perform transactions

§         Relationship: as a tool to improve relationship with users

Within the categorization above, a second dimension of the services is also taken into account to determine the degree of adherence of the services in each category to interaction models commonly used in the Web. This second dimension subdivides each of the three categories above into: (i) Basic services (least changed as compared to channels other than the Web); (ii) Intermediate services (adopting some Web elements but not entirely innovative); and (iii) Advanced services (specifically developed for the Web environment). Table 2 summarizes the division of the categories above according to the different interaction levels for each service class.Table 2: Categorization of website services

(Source: Diniz, 2002)

4.        The reliability dimension

More than a purely technical matter, many experts regard security as a matter of customer perception (Albertin, 1999). As in conventional transaction environments, the digital world too security is never an absolute (Camp, 2000). Transaction errors may occur, be it out of bad faith, user naiveté or lack of expertise, misconduct, fraud, theft, aggression or trespass by third parties. In addition to straightforward human acts, security problems can arise from systems failures or even from the organizational process itself.

Schneier (2000) states that Internet is probably the most complex system ever developed, since it is a public network with millions of computers connected to a highly complex physical network. Each of these interconnected computers has hundreds of software running, and some of these interact with other programs, either in the same computer or in another one connected to the network. As a result of its complexity, Internet security relates with the prevention and detection of, and reaction to, trespass, fraud and loss to prevent financial and moral damages.

The best information security management practices code was developed by BS ISO/IEC 17799 (2000), and is characterized by the preservation of:

§         Integrity: data can not be corrupted during handling or transmission.

§         Confidentiality: data can not be handled or read by unauthorized people.

§         Availability: communication between computers must take place whenever requested.

For the purposes of the Internet, other information security checks have been added (Camp, 2000):

§         Privacy: concerning disclosure of or access to information through electronic means.

§         Non-repudiation: ensuring that a completed transaction cannot be denied.

§         Authenticity: ensuring that the signatory of a document is really who they purport to be.

Considering the use of computer science itself (Meirelles 1994) and the increasingly disseminated use of technology (Schneier 2000), the security process may also be divided into three layers: physical, logical and human.

The physical layer is characterized by the location of the hardware. It is the physical space where users interacts with the Internet, where the means of communication (cable and waves) are, and where the data is physically stored.

Table 3: Security Items X Security Layers

The logical layer is characterized by software-based protection. This layer is defined by encryption and decryption solutions, presence or lack of digital certification of computers and users, application development language, databases, communication middleware with both legacy systems and new ones.

The human layer is characterized by human behavior in the use or maintenance of information systems. According to Schneier (2000), this is the weakest link in the security chain, and chronically responsible for most security system failures. The most important aspect of this layer is people’s perception of risk: how they deal with events that rarely occur; whether they are skilled users or not; the hazard of malicious or unwitting trespassers; and the social engineering by means of which hackers can legally obtain information (Schneier 2000).

Table 3 summarizes the six security items as they manifest in the three layers described earlier. Items that can be identified by simple browsing are highlighted in bold typeface. This table will be the basis to understanding the items that have to be considered for the purposes of evaluating the reliability of a business transaction environment on the Internet.

5.        The usability dimension       

Since computers are available to a great number of people and support a great variety of applications, information systems projects increasingly require accessibility, usability and user involvement (Laurel and Mountford 1990). The priority for interface designers must be to improve computer usage, conceiving ever more intuitive and user-friendly systems (Rheingold, 1990; Lees 2002).

Usability has become even more important in the Internet age (Nielsen 2000) since it takes place before customers spend any money on potential purchases: on the Web, users experience usability first and pay later. A study of e-commerce usability (Nielsen et al. 2001) found a success rate of only 64%, meaning that more than one-third of all attempted e-commerce transactions fail to reach completion. Evaluation of the causes for this revealed that about two-thirds of the problems in online transactions can be traced back to poor usability projects. Figure 2 shows the more frequent causes of failure found at the sites studied.

Figure 2: Reasons for transaction failures in digital environments (Source: Nielsen, 2001)

Non-functional usability requirements include interface quality, user-friendliness and human factors often overlooked by software engineers (Ferreira and Leite, 2002). The usability evaluation model used here was developed based on three categories (Pressman, 1997):

§         Layout: concerns display of information on the Web page

§         Data entry: concerns fields for user information collection

§         User on command: concerns how much control users are given over the page

For each of these categories, a group of items was defined to enable ascertaining the Website’s usability level. As for ‘Layout’, three items were selected as being helpful towards Internet banking sites evaluations: consistency, visibility and clarity. Three items were also selected for the ‘Data entry’ category: forgiveness, feedback and error treatment. And a further three items were selected for the ‘User on command’ category: search tools and links, constant text size, and site map. The explanation for these items can be found in systems design literature (Microsoft 1991; Apple 1992; Tognazzini 1990; Pressman 1997; Ferreira and Leite 2002; Shneiderman 1998; and others). Table 4 summarizes these three categories and their items.

Table 4: Categories used to analyze the Usability Dimension

6.        Evaluation of Internet banking sites

Three banks in Brazil were selected for Internet banking services testing on the model’s three dimensions: Bradesco, BankBoston and Banco do Brasil. The choice is justified by the specific importance each has for the use of Internet banking in Brazil. Bradesco was the first Brazilian Website and the fifth in the world to offer banking services over the Internet (Gates 1999). Banco do Brasil (BB) is the largest bank in the nation and competes with Bradesco for leadership in absolute number of Internet banking users (Business Standard, 2002). BankBoston (BKB), although not as large as the other two, has the lead Internet banking usage ratio, with 42% of its customers making use of the bank’s Web-based services (Hessel 2003). Furthermore, this sample includes one bank from each segment of the industry: one private Brazilian bank (Bradesco), one state-controlled bank (Banco do Brasil) and one foreign bank (BankBoston) with operations in Brazil, therefore encompassing different experiences and Internet banking strategies.

Besides the fact that these are three of the most important institutions in the Brazilian banking sector, this sample is also convenient because since each one of the authors is a customer of at least two of these banks, they have full access to the Internet banking part of the Web sites.

Based on a check list previously prepared with all features described in the three dimensions, each author/customer individually accessed the Internet banking services of two different banks, to ensure a double checking process for all of them. After the data gathering process, a meeting was organized to discuss the individual findings, adjust and refine the data, and deal with any possible doubts relative to application of the criteria. Following are the findings of the evaluation done on the three Internet banking sites.

6.1         Functionality

The functionality dimension is usually identified with the offer of bank services, and the three banks at hand (Bradesco, Banco do Brasil and BankBoston) offer a large roll of services. To consider the functionality dimension for these three banks, we may use the results published by Business Standard Magazine (2002 and 2003), showing a ranking of Internet banking services in Brazil for the past two years, with criteria based on the model illustrated in Table 1. Bradesco, second best in individual services in 2002, maintained its position in 2003. Banco do Brasil, fourth place in individual services in 2002, dropped to fifth in 2003. BankBoston, with the lowest functionality ratio of the three banks considered, ranked 14^th in 2002 and climbed to 12^th in 2003.

Internet banking functionality is similar at the banks researched: all three post information at the basic and intermediate levels, and customization is already a standard at the advanced level. As for the transaction category, Bradesco offers a greater list of services, followed by Banco do Brasil. On the relationship category, Banco do Brasil does a better job of using the Web to communicate with customers, offering even an on-line consulting channel.

6.2         Reliability

At the logical layer, item ‘integrity’, BankBoston and Banco do Brasil, the certified access option include the hash function, which generates an algorithm that checks for document integrity and enables determining whether or not the document was corrupted during its transit over the network.

Confidentiality and privacy are ensured at the three banks by means of encryption, which varies in degree according to the technology used. The simplest one is Bradesco’s and the most complex is Banco do Brasil’s under the certified access option. As for the non-repudiation and authenticity item, BankBoston and Banco do Brasil, under the certified access option, use digital certification by the customer, and the entire process takes place through asymmetric encryption. Concerning these two later items, the technology used is determined by reading the explanatory texts displayed upon access to the online services and by installing a security component. The difference in use of asymmetric encryption between BankBoston and Banco do Brasil is that BankBoston is the issuer of the certificates, and access to the security component is not controlled, while Banco do Brasil’s certificates are issued by Certisign and download of the security component is restricted to those customers who submitted a signed agreement at the bank branch.

At the physical layer, Bradesco was the first bank in Brazil to use a virtual keyboard to ensure secrecy of the access password, but errs by not making use of the virtual keyboard mandatory. Banco do Brasil also uses virtual keyboards, and only through these can the access password be entered.

At the human layer, confidentiality, privacy and non-repudiation are ensured at all three banks by means of a terms of use agreement or by a letter of agreement under which customers are bound to perform the registration procedures. At Banco do Brasil, issuance of the certificate is only authorized after confirmed signature of the letter of agreement at the bank branch. Banco do Brasil and BankBoston disclose their privacy policies at their Websites. Bradesco’s security policy is limited to the procedures to gain access to the online services, and a session called “security information” that provides information on on-line security and protection. Confidentiality and privacy are also guaranteed by a bill currently under legislative appreciation.

Authenticity at the human layer is ensured by registering access passwords, in addition to the type of data and information requested at the time of registration. Each bank has its own flow and criteria, based on the ATM card passwords and and call center access PIN. The complexity of registration is not necessarily related to the quality of the security. The complexity or simplicity of the process may or may not simplify adhesion to and use of the service, and may be remedied by a clear, easily understood communication at each bank’s Website.

6.3         Usability

As regards layout, at Banco do Brasil the various services are organized hierarchically, and can be found easily by placing cursor upon the tool bar listing the main service groups (also called ‘subsites’): balances; transfers; payments; etc. At Bradesco the second bar (subsites) only shows the services available via Internet Banking and its information is hierarchically structured. The service in use is, however, somewhat difficult to locate, as the subsite title is not highlighted. At BankBoston, services hierarchy is displayed by running the cursor through the links of the ‘Search’ section.

Banco do Brasil’s Internet Banking offers links to BBresponde (help desk) and to FAQ in all pages. BankBoston offers a ‘Menu’ link that drives users on to Internet Banking services. All of Bradesco’s pages offer a site map.

‘Account statements’, the most popular Internet banking service, was chosen to analyze data entry usability at the three Banks. Banco do Brasil asks for the relevant statement month but, in some situations, the current month’s statement is provided in the absence of this information. This is inconsistent with user needs on the first day of any given month. At Bradesco balances can be seen at a specific subsite (‘Account Balances and Statements’) in the upper-left corner which covers checking and saving accounts with the possibility of choosing from five different periods. At BankBoston, customers are only allowed to enter the initial date of the statement, which makes balancing difficult because the covered period is non-customizable. BankBoston Internet Banking uses the metaphor ‘calendar’ for all services that require the filling in the ‘date’ field (payments, transfers, etc.) and follows the consistency principle documented by Tognazzini (1990), according to which mechanisms must be used in the same manner, no matter when and where they occur.

Banco do Brasil’s and Bradesco’s Internet Banking Websites  offer no open-ended search, only a menu with a list of predefined options. At the sites of all three banks, when an error occur, a warning appears, in most cases advising how to correct it.

Banco do Brasil offers a customization option at the ‘My page’ link, allowing users to define how they prefer to visualize the more relevant information based on a minimum configuration. The ‘change text size’ option has no effect on the Internet Banking pages. Bradesco Internet Banking text size can be changed by the customer, by means of browser commands. This is an important aspect to meet the visual needs of different customers using a service whose major feature is numbers layout.

7.        Conclusion

This article sought to show the use of a model for evaluating a virtual business environment based on three dimensions: Functionality, Reliability and Usability. We believe that these dimensions represent fundamental elements for the conduction of transactions, and that they should also be considered for the evaluation of virtual business environments.

As stated under the ‘Functionality’ dimension, most of the Basic services are available at the sites of Banco do Brasil, Bradesco and BankBoston. However, the adoption of Intermediate level services demands a higher level of user-Website interaction, compelling organizations to regard the digital channel as an environment with specific needs that requires reorganizing internal procedures. Since this involves resources that demand a specific Web technology skills, a better mastery of the technology and control of its consequences will be required. At an Advanced level, the use of technology will only yield results where perfectly integrated with the organization’s structure. The demand created by the level of interaction also has an impact on other areas, necessitating strategic redefinition and affecting internal organization flows.

Reliability of a service may be incremented by using a larger number of resources to make sure the six security items (integrity, confidentiality, availability, privacy, non-repudiation and authenticity) are present at the three layers (physical, logical and human). There must be a cost/benefit analysis of adopting a larger number of resources versus the complexity of the process and the implementation and maintenance costs of the security architecture. The services analyzed at the three banks boast security architectures that guarantee reliability at a level sufficient to satisfy a large and heterogeneous public. But Internet Banking still has some weaknesses as regards the ‘non-repudiation’ and ‘client authenticity’ items.

Concerning Usability, the ‘Layout’ and ‘Data Entry’ categories meet most of the requisites proposed by those involved.  In certain cases, details are overlooked by those responsible for the website, making browsing less intuitive and the interface not as user-friendly as it might be. On the other hand, all three banks should make their Internet Banking services more flexible, so that customers might customize it according to their tastes. Adding user-friendliness, reliability and additional services to the Website is the best way to turn customers who check their balances and account statements into Internet Banking-traders.

Due to the dynamic nature of the Internet business environment, this 3-dimensional evaluation model the corresponding criteria do not purport to be final, and will require constant improvement. In the future we expect to apply this model to other virtual business environments in order to reach a refined model that may effectively serve as an evaluation  tool.

References

• “Macintosh Human Interface Guidelines” (1992) Apple Computer. Addison-Wesley Publishing Company.

• “The Windows interface: an application design guide”. (1991) Microsoft Programming Series. Microsoft Corporation.

• Albertin, Alberto L. (1999) Comércio eletrônico: modelo, aspectos e contribuições de sua aplicação. Ed. Atlas. São Paulo.

• Angulo, Marcelo J. et Albertin, Alberto L. (2000) “Portais ou labirintos?” Anais ENANPAD 2000

• Barnes, Stuart I.; Vidgen, Richard. (2001) An Evaluation of Cyber-Bookshops: The WebQual Method. International Journal of Electronic Commerce Vol. 6 Issue 1, Fall.

• BS ISO/IEC 17799. (2000) “Information technology: Code of practice for information security management”

• Camp L. Jean (2000) “Trust and Risk in Internet Commerce”. MIT PRESS.

• Cano, Carlos B. et Becker, João L. (1999) “Organizações que operam no espaço cibernético: um estudo de caso comparativo entre o serviço de busca Altavista e a Amazon Books”. Anais ENANPAD.1999.

• CIAB (2003) “Bancos, tecnologia e modernização da sociedade” XIII Congresso Internacional de Automação Bancária. Febraban. tecnologia e modernização da sociedade. Febraban, 2003.

• Cunha, Maria Alexandra A V. C. et Reinhard, Nicolau. (2001) “Portal de Serviços Públicos e de Informação ao Cidadão: Estudo de Casos no Brasil”. Anais ENANPAD 2001        

• D’Andréa, Edgar et al.( 2000) “Segurança em banco eletrônico”. PricewaterhouseCoopers.

• Davis, Fred D. (1989) Perceived usefulness, perceived ease of use and user acceptance of information technology. MIS Quarterly, vol. 13, iss. 3, Sep.

• Diniz, Eduardo H, Porto, Roseli M., Ângulo, Marcelo J. (2002) “Strategic Differences on Web Use by Corporate and Retail Banks” Balas 2002 Proceedings.

• Diniz, Eduardo H. (2004) “Evolução e Segmentação no Perfil dos Serviços Bancários pela Internet” Relatório de Pesquisa NPP Nº 40. 112 p.

• Erickson, Thomas D. (1990) “Working with interface metaphors” in The Art of Human-Computer Interface Design – Edited by Brenda Laurel. Addison-Wesley Publishing Company.

• Ferreira, Simone B.L. et Leite, Júlio Cesar S. P. (2002) “Exemplificando aspectos de usabilidade em sistemas de informação”. Anais ENANPAD 2002

• Freitas, Henrique M.R., Janissek, Raquel, Luciano, Edimara M. et Oliveira, Mirian. (2001) “Observando a prática de negócios na Internet: os casos do Submarino.com e Lokau.com” Anais ENANPAD 2001

• Gates, Bill. (1999) Business at The Speed of Thought. Warner Books.

• Heidjen, Hans van der. (2000) e-TAM: a revision of the technology acceptance model to explain website revisits. Research Memorandum, vol. 29, Sep.

• Hessel, Rosana. (2003) “Brasil já é o segundo do mundo em Internet banking”. Gazeta Mercantil, Caderno Tecnologia de Informação. 1/abr/2003.

• Horton, Robin P.; BUCK, Tamsin; WATERSON, Patrick E.; CLEGG, Chris W (2001). Explaining intranet use with the technology acceptance model. Journal of Information Technology, Vol. 16 Issue 4, Dec.

• Laurel, Brenda et Mountford, S. Joy. The Art of Human-Computer Interface Design – Edited by Brenda Laurel. Addison-Wesley Publishing Company, 1990

• Lederer, Albert L.; Maupin, Donna J. (2000) The technology acceptance model and the World Wide Web. Decision Support Systems, Vol. 29 Issue 3, Oct.

• Lees, Jeff. (2002) “RBC Royal Bank´s Online Banking Initiatives: usable design now and for the future.” White paper – Human Factors International.

• Loiacono, Eleanor; Watson, Richard T.; Goodhue, Dale L. WebQual: a measure of Web site quality. Working Paper Terry College of Business. From www.terry.uga.edu/cisl/includes/pdf/webqual.pdf, accessed in September 2003.

• Mariano, Sandra Regina H., Mayer, Verônica F., Ferreira, Patrícia T.F. et Gomes, Luiz Flávio A M. 2002 “Aplicação de métodos de apoio à tomada de decisão multiatributo na avaliação de sites de varejo brasileiros” Anais ENANPAD 2002

• Meirelles, Fernando S. (1994) Informática: novas aplicações com microcomputadores. Editora Makron Books.

• Mendoza, Luis E.; Griman, Anna C.; Perez, Maria A.; Rojas, Teresita (2002) Evaluation of environments for portals development: a case study. Information Systems Management, , Vol. 19 Issue 2, p70, Spring.

• Moon, Ji-Won; Kim, Young-Gul. (2001) Extending the TAM for a World-Wide-Web context. Information & Management, Vol. 38 Issue 4, Feb.

• Nielsen, Jakob et Tahir, Marie. (2002) Homepage Usabilidade: 50 websites desconstruidos Editora Campus. Rio de Janeiro.

• Nielsen, Jakob. (2000) Designing Web Usability: The Practice of Simplicity. New Riders Publishing.

• Nielsen, Jakob; Molich, Rolf; Snyder, Carolyn; Farrell, Susan. (2001) E-Commerce user experience. Nielsen Norman Group.

• Nogueira, Antonio R. R., Soares, Cintia M et Portela, Luiz M.Q. (2000) “A definição dos negócios em comércio eletrônico e a questão das funcionalidades de navegação: um estudo em sites brasileiros” Anais ENANPAD 2000.

• Norman, Donald A. (2002) The Design of Everyday Things. Basic Books,

• Oliveira, Mirian. Fatores internos de atração: o que adotar no Website. Anais ENANPAD 2002

• Pressman, Roger S. (1997) Software Engineering - A Practioner’s Approach – 4th ed., McGraw-Hill, Inc.

• Ramos, Anatália S. M. et COSTA, Fabrício S. P. H. A. R. (1999) “Serviços bancários pela Internet: uma proposta de avaliação integrada de competidores e clientes”. Anais ENANPAD 1999.

• Ramos, Anatália S. M. et VELDMAN, Selma M S. (2000) “Relacionamento com o cliente através da Web: um estudo teórico-exploratório”. Anais ENANPAD 2000.

• Revista Business Standard. (2002) 25 Melhores Serviços de Internet Banking. Edição especial, IDG – Computerworld do Brasil, April/2002.

• Revista Business Standard. (2003) 25 Melhores Serviços de Internet Banking. Edição especial, IDG – Computerworld do Brasil, Junho/2003.

• Rheingold, Howard. (1990) “Interview with Don Norman” in The Art of Human-Computer Interface Design – Edited by Brenda Laurel. Addison-Wesley Publishing Company.

• Sartori, Tubino et Pereira. (2002) “Utilização da Internet pelas empresas de Maringá sob a ótica de Websites”. Anais ENANPAD 2002

• Schneier, Bruce. (2001) Segurança.com - segredos e mentiras sobre a proteção na vida digital. Campus. Rio de Janeiro.

• Shneiderman, Ben. (1998). Designing the user interface: strategies for effective human-computer interaction. 3^rd ed. Addison Wesley Longman.

• Soares, Rodrigo O. et Hoppen, Norberto. (1998) “Aspectos do uso da Internet nos negócios pelas grandes empresas no Brasil: um estudo exploratório baseado em sites Web”. Anais ENANPAD 1998.

• Steil, Andréa, Pacheco, Roberto C.S. et Barcia, Ricardo M. (2001) “Uma Pesquisa Sobre os Atributos da Interface das Organizações Virtualizadas no Cenário Brasileiro”. Anais ENANPAD 2001

• Tognazzini, Bruce. (1990) “Consistency” in The Art of Human-Computer Interface Design – Edited by Brenda Laurel. Addison-Wesley Publishing Company.

• Vertelney, Laurie et Booker, Sue. (1990) “Designing the Whole-product User Interface” in The Art of Human-Computer Interface Design – Edited by Brenda Laurel. Addison-Wesley Publishing Company.

• Zeithaml Valarie.A.; Parasuraman, A.; Malhotra, Arvind (2002) Service quality delivery through Web sites: a critical review of extant knowledge. Journal of the Academy of Marketing Science, vol.30, iss.4.

EJISE is published by Academic Conferences Limited
Curtis Farm, Kidmore End, Nr Reading RG4 9AY, England
Tel: +44 (0)1189 724148, Fax: +44 (0)1189 724691, Email: info@ejise.com

Send mail to info@academic-conferences.com with questions or comments about this web site.
Copyright © 2002-2006 Electronic Journal of Information Systems Evaluation
Last modified: September 29, 2005
ISSN 1566-6379