Website Testing

Test Levels for Web-based applications By G. Bazzana, F. Basili, E. Fagnoni

The last years have seen an explosive growth in the WWW. Currently the Web is the most popular and fastest growing information system deployed on the Internet, representing more than 80% of its traffic. Additional trends are: of Web-based solutions with large DBMS; ‹Web-portals; ‹ Usage of Web-based interfaces for ‹ Intranet/ Extranet applications that directly interface the company legacy system; ‹Usage of Web-based approaches for critical applications (e.g.: on-line trading) ‹Access to the Web by different media (e.g.: mobile phones, TV) ‹Need to allow equal opportunities to Web access also for impaired or disabled people, in order not to exclude them from the new “Information Society”. ‹Interaction

This has increased the complexity and criticality of applications, requiring the adoption of systematic testing activities also in the Web-based realm that is far too often wrongly considered an application domain populated mostly by hackers. The increasing importance and reliance of Web applications ask therefore for more and more testing levels to be applied, as highlighted in the lower part of the picture.

To date, we can say that Web-based applications deserve a high level of all software quality characteristics defined in the ISO 9126 standard, namely: ‹Functionality:

Verified content of Web must be ensured as well as fitness for intended purpose ‹Reliability: Security and availability are of utmost importance especially for applications that required trusted transactions or that must exclude the possibility that information is tampered ‹Efficiency: Response times are one of the success criteria for on-line services ‹Usability: High user satisfaction is the basis for success ‹ Portability: Platform independence must be ensured at client level ‹Maintainability: High evolution speed of services (a “Web Year” normally lasts a couple of months) requires that applications can be evolved very quickly. In the experiences of the authors, Webbased applications are characterised by the following project management peculiarities: ‹Development is managed in accordance with Rapid Application Development (RAD) approach; ‹This implies that analysis and design are scarce when compared to “standard” applications: the goal is to sketch-out an innovative idea into a service, rather than to build a product starting from very precise specifications and architectural design ‹Proof of concept presentations are the

normal way to set-up “live” specifications ‹Limited formalisation of analysis and design automatically implies that usage of defect prevention techniques can only be marginal: most of things to be checked are thus let to dynamic testing ‹Round-trip engineering is followed, by which we do not have a waterfall model but rather we: “design a little, implement a little, test a little” several times on incremental versions; Such characteristics have marked the success of the Web; hence we do not think that Web development has to be adjusted in order to fulfil traditional software engineering practices, but rather testing techniques and tools have to be capable of operating within such innovative approach. The following aspects further complicate the picture: ‹Designers are often not professional software developers or at least are not aligned with conventional software engineering practices; ‹The turnaround of people involved in Web projects is extremely high ‹Compressed deadlines for services are normal considering the pace of innovation in the field; ‹Evolutionary maintenance is a must; we can say that Web-based products seldom reach a mature status, since they are replaced by newer version when they still are beta-tested; ‹Underpinning technologies are chang-

site are in relative ../images/image.gif),...).

forms

(e.g.

3. Usability tests are concerning with the following aspects with respect to normal behavior, destructive behavior, and inexperienced users: ‹Coherence of look and feel ‹navigational aids; ‹user interactions; ‹help messages; 4. Fast loading/link tests are concerning with the following problems: ‹home page weight should be less than 45k; ‹every page weight should be less than 50k; ‹every web-site should have a fast loading abstract/index; ‹every IMG tags must have WIDTH and HEIGHT attribute. Fast loading test is very important if we consider that 85% of Web Users indicate slow loading times as the reason for avoiding further visits to Web Sites. Moreover, a survey made by Zona Research Inc. in April 1999 highlighted very high bail-out rates for pages with a weight resulting in more than 8 seconds to wait;1 second load-time improvement brought to a reduction in bail-out rates from 30% to 7% ! ONION static testing checklist In order to have a pragmatic approach to Web syntactic testing, a standard checklist, containing more than 100 checks, was devised by ONION to be applied both for acceptance purposes and for regression testing activities. This checklist covers the following aspects (for each class the number of tests is given, together with some of the aspects checked): ‹stylistic problems (9 tests, including: spelling errors, particular tags, use of obsolete mark-up, particular content-free expression, empty container elements, etc.), ‹lexical problems (5 tests, including: use of character sets, formatting-related problems, using white spaces around element tags, etc.), ‹syntax problems (12 tests, including: illegal elements, illegal attributes, unclosed container elements, malformed URLs and attribute values, etc.), ‹fast loading related problems (26 tests, including: bandwidth consumption, images syntax, etc.), ‹document structure problems (4 tests

applicable both to tables and forms), ‹portability problems (17 tests including: accessibility by various browsers and platforms, mark-up inside comments, use of single quotation marks for attribute value, use of specific mark-up not supported by all browsers, liberal usage of file naming, etc.), ‹structural integrity problems (4 tests including: no index file for a directory, dead links, limbo pages, etc.), ‹security problems (7 tests including: no confidential data passed through form without SSL, no user form field exposed to shell in CGI programs, etc.) The adoption of supporting tools can allow for setting up a test factory and thus run almost automatically a large proportion of the defined tests.

Tools for Testing Static Web Applications In this section, the most known tools for Testing Web Static Applications are quickly examined. This does not imply any endorsement by the authors of any of the listed tools. Information on listed tools might not be always fully aligned with their latest version, owing to the fast evolution pace. In general we can say that no one tool is fully comprehensive in its coverage, and is best used in combination with additional tools. For most tools, coverage improves with every release The World-Wide Web Consortium (W3C), hosted by MIT, INRIA and University of Keyo, has been committed from its beginning, under the leadership of Tim Berners-Lee, the inventor of the WWW, to developing a neutral, open forum for the evolution of Web Technology. Like its partner standard body, the Internet Engineering Task Force (IETF) W3C is committed to developing open, technically sound specifications, backed by running sample code. As a consequence W3C has developed various tools for Web Testing including: ‹ An HTML Validator, which allows HTML documents to be validated against the DTDs for HTML, including HTML 4.0 ‹A CSS Validator, which allows the user to validate the CSS style sheets used by HTML and XML pages

‹HTML Tidy,

a free utility for correcting HTML syntax automatically and producing clean mark-up. Tidy can be used to convert existing HTML content into compliant XML All W3C software is Open Source and can be retrieved from http://www.w3.org/

Doctor HTML Doctor HTML is a Web site analysis product, whose main features are: ‹Check the document for spelling errors: This test looks for spelling errors in the document. This is very helpful because it removes HTML directives and accented text before running the document through a spelling checker, eliminating most of the false alarms; ‹Perform an analysis of the images: This section loads all the images in a document and determines a few important properties of each image. The most important information is the bandwidth consumed by each image, and roughly how long it will take to download over a 14.4kbps modem (now the most common speed for dial-up access users). Excessive load times for individual images are highlighted. The program also reports the size and number of colors in the image, which has a direct bearing on how much bandwidth the image consumes; ‹Test the document structure: This feature tests the main document structure (excluding tables, which are dealt with separately). The test looks for unclosed HTML codes that may cause problems on some browsers. When used in conjunction with other features, this can be helpful in hunting down extra HTML tags; ‹Look at image syntax: This test deals with one of the most common mistakes in HTML coding: overlooked image command tags. Specifically, it checks each image command for HEIGHT, WIDTH and ALT tags, and reports if they are absent. These tags are important for quick image loading and page formatting, as well as providing information for browsers lacking images; ‹Examine table structure: This feature tests the table structure on the page. It specifically looks for unclosed TR, TH and TD tags inside a properly defined table (e.g.: one which has both an open and close table tag). It also reports on TR, TH and TD tags that appear outside of any properly defined table, since these may cause formatting errors on some browsers; ‹Verify that all hyper-links are valid:

long time on the market for C language. Among the various development/ test environments offering such features it is worth remembering at least: SUN’s Java Test Tools, TCAT for Java, White Box Deep Cover for Java and RST Test tools (inclusive of: Deep Cover for coverage analysis; Assert Mate for pre-conditions, post-conditions and data assertions testing; Total Metric for static analysis).

Server Extension

Testing of dynamic Webbased applications

DBM

Module testing Testing of Dynamic Web-based applications deserves much of the challenges of client-server applications, with additional constraints posed by the underlying architecture, which can be summarised as represented by Figure 2. Dynamic WWW development can be done, at the current level of technology, with two main approaches: CGI Programming (Perl, C, TCL..) or Server Extension Programming (ASP, PHP, Apache Server API, Netscape Server API, ..) It has to be noted that the risk level associated to the two techniques is utterly different, as clearly highlighted by Figures 3 and 4. In fact, whereas when a CGI calls fail just a program fails, when a server extension fails, the whole server might crash! For Testing Server Side, an approach similar to client-server testing has to be taken, covering both GUI and HTTP testing, as shown in Figures 5 and 6. Besides the well known techniques for client-server testing, you should beware of complexity from included software layers; it has to be remembered that often Figure 3: CGI Programming

MIS

CGI

Transaction Mng

Figure 2: Architecture & Challenges of Dynamic Web Applications more than 90% of the software is out of the developers’ control, being re-used from other sources. Special care has to be devoted to the following aspects: ‹Focus on usability test. ‹Focus on performance test. ‹Focus on load-stress. ‹Focus on installation test if special plug-ins are present ‹Perform well-managed beta test.

Integration/ Security Testing Integration/Security tests are concerned with ‹Included components ‹Cookies ‹Proxies/ caching ‹Frames

be read back from the browser. This is useful for having the browser remember some specific information. The problems related with testing are that cookies expire and that users can disable them in browser. Besides the basic Security checking performed during the previous test level, specific security testing has to be performed when Web applications make usage of sensitive data. First of all it shall be clear that on the WWW there is no silver bullet for absolute security, likewise in real life, and that security techniques and checks shall be tailored depending on the value to be protected. Moreover, it is important to underline that security enforcing involves both organisational and technical issues; namely organisational issues are often much more important than technical ones, at least in Intranet and Extranet applications.

A peculiar aspect of the testing of dynamic WWW applications is testing of cookies. The web is a memory-less system, with no concept of session. To overcome these issues you can use cookies: a small In such cases, the approach is to define a piece of information sent by a web server “Security Policy” at company level and to store on a web browser so it can later Figure 4: Server Extension Programming

HTTP SERVER CGI interface CGI PROGRAM

DB

Browser

WEB SERVER

LEGACY APPLICATION

virtual users; HTTP requests can be edited, changed or parameterised ‹Web Scenario Wizard: orchestrates virtual users into a multi-user scenario; up to 4.3 million hits per day like real users surfing ‹Visual Load Testing Controller: drives, monitors and synchronises interactions Data analysis statistics include number of virtual users, transaction performance, completed transactions per sec., connections per sec., and throughput. Web-specific supporting features include: cookies, proxy servers, user authentication, session Ids, CGI scripts, API calls, and HTML forms. The tool can be integrated with Astra Site Manager and SSL Plug-in Pack.

Testing ERP and WWW integration Future Intranet/ Extranet applications will require more significant work and more sophisticated skill set. In fact, Intranets will evolve into a component of the IT infrastructure making distributed computing more open, simpler and more manageable. This will make possible the delivery of more flexible, manageable distributed business processes. From a technical point of view, Webenabled business applications will be based on transaction-oriented business processes; hence Intranet based applications will merge with Extranet-based business-to-business transactions, EDI and electronic commerce transactions. Already today, to multiply benefits, companies need to integrate Web technology with transaction-oriented business applications, group-ware and infrastructure services, integrating Web-based application and MIS and setting-up simple, cross-platform applications on top of a simple-to-manage and more centralised IT infrastructure. As far as testing is concerned, challenges are on: ‹security, ‹load testing ‹User authentication ‹Server authentication ‹Connection privacy ‹Message Integrity ‹Payment security.

Tool providers are performing certified integration with ERP systems.

Future challenges Future challenges are related to the evolution of the WWW, namely: ‹ New Generation HTTP Protocol (HTTP-NG) ‹Integration between TV and the Web ‹Emergence of XML ‹New references for User Interfaces: evolution of HTML (for publishing documents), on MathML (for publishing Math), on SMIL (for multimedia presentation), on SVG (for publishing diagrams and vector-based graphics) ‹Mobile access to the WWW as well as techniques for using voice interaction for accessing the Web ‹Privacy issues ‹Digital signatures ‹Micro-payments These emerging technologies and services will require Internet testing approaches to be continually fine-tuned, to guarantee the reliability and quality of service required by the global Information Society. ONION S.p.A. Via L. Gussalli, 9 - 25131 BRESCIA (Italy) E-mail: [email protected] Web: http://net.onion.it/