Interfaces

Digg This!

Page 1 of 2   next page »

The advent and rapid adoption of AJAX technology by the Web community has truly revolutionized the user experience as well as the development of Web applications. AJAX overcomes the technical limitations of classic post->submit->reload CGI applications. So one can implement complex and advanced tasks as Web applications with an appealing user interface. The result is that today's AJAX Web applications don't have much in common with the old-style Web applications anymore but are on a par with classic desktop GUI applications in terms of sophistication. Combined with the Web's network transparency and easy deployment opportunities the result is a development environment with hard-to-beat power.

Many Web technology vendors have understood this trend and have created and marketed powerful Web client frameworks. By using these toolkits it's simpler nowadays to create a Web application with an appealing GUI than to create a classic desktop GUI application.

This development now asks for new solutions for a vital part of the software development process: testing the new-style user interfaces - ideally automated testing. It doesn't suffice anymore to test the back-end and run only unit-tests. The user interface has to be tested as well to provide high-quality applications.

While engineers testing AJAX GUI applications principally face the same kind of challenges as when testing classing GUI applications, the dynamics and networked nature of AJAX applications as well as the variety of Web toolkits, available Web browsers, and operating systems pose additional challenges.

This article will discuss the motivation for automating user interface tests and the challenges and solutions of creating automated tests for AJAX applications. The material is based on the collected experience of providing consulting and support services in real-world automated testing scenarios.

Motivation
First, let's look at the motivation for automating the testing of the user interface.

• Faster: Once automated tests are available, it's much faster to execute the tests than when doing this manually. More tests can be executed at the same time and run more often (like every night). This way regressions are found quicker after they're introduced. This makes finding and fixing the bugs that lead to the regression easier. Bottom line, we save time, money, and nerves.
• Trivial to replay in different configurations: Once a test is automated and the right tool has been chosen for the task, there's no additional effort to run a test in different configurations (different browser, OS, hardware, etc.). A good Web application should work on at least the most common browsers such as IE 6, IE 7, Firefox 1.x, Firefox 2.x, and Safari on Windows, Linux, and Mac OS X. This makes nine configurations in total, so testing manually has an overhead of 800% over testing automatically.
• Reliable and reproducible: Good programmers and test engineers strive to work efficiently. This means they optimize monotonous tasks. Normally, this is a good thing since this way innovative solutions are created. When executing a manual test, optimization is a bad thing. It's desirable to execute exactly the same steps every time. Otherwise issues and problems won't be reproducible, or even worse, not found at all. Humans aren't suited to running the same tests over and over again.

  Computers are the exact opposite. They're good at executing the same over and over again. They're more reliable testers than humans.

• Relieve testers from monotonous tasks: When automating a test, test engineers don't have to do the boring job of executing the same manual tests all the time. Instead they can work on interesting tasks such as creating new tests, setting up automation frameworks, and analyzing the results that provide additional motivation.

While all this clearly speaks to test automation, one should never expect an automated testing tool to do all the work for you. Using an automated testing tool you can't get rid of your testers. At least initially some work has to be done to create the tests and set up the framework. Tool vendors who claim that their products will automatically create all the tests for you are naive or guilty of negligent misrepresentation.

There will always be a few tests that simply require too much effort to automate. Checking a printout of the application is something that you probably might still want to test, for example. So the goal of 100% test automation isn't feasible.

Automating most tests will deliver software of a higher quality while reducing costs. It'll get you faster releases by shortening the long manual test and release cycles.

Challenges and Solutions
This section will discuss the main challenges and solutions when setting up automated UI tests for AJAX applications. Common mistakes will be covered as well.

Robustness and Object Identification
The main requirement for an automated test is that it has to be robust and not break when the application being tested changes. Neglecting this important requirement is one of the main reasons for failure of many automated UI testing efforts.

To create a robust test, a test script has to interact with the AUT (application under test) in an abstract way. This means that in emulating a click, for example, on a tree item in an AJAX tree control, it isn't good enough to click to a fixed coordinate (x, y) in the Web page. It isn't even good enough to click a specific DOM element (e.g., a DIV containing the tree item representation) without knowing more. Both ways, the test will break quite easily if the position, representation, implementation, or structure of the Web page or tree control changes.

To overcome this, a testing tool has to fulfill the following requirements:

• Toolkit Knowledge: A testing tool has to recognize high-level controls (tree, table, layout controls, date picker, input field, buttons, menus, etc.) and provide an interface to interact with the controls and query their states and properties on an abstract level. This requires knowing about the Web client toolkit built into the testing tool but as a result, robust tests can be written.

  One special challenge in the AJAX world is the myriad open source and commercial Web client toolkits each providing their own special implementation of advanced controls such as layout controls, tables, trees, menus, etc. - not even counting the high number of unknown internal Web client frameworks.

  It's highly unlikely that there will ever be an automated testing tool that will support all of these frameworks out-of-the-box. Does this mean that it's impossible to create robust automated tests for your application if you use a Web client toolkit that isn't supported by a testing tool?

  The answer is: No, it's still possible.

  When evaluating Web testing tools it's advisable for you to find out if it's possible to extend the framework yourself or, even better, if the vendor is willing to extend the testing tool to recognize and support the controls you're using. So even if your Web client toolkit isn't supported out-of-the-box, it just needs a way for extension to create robust automated tests.

  This means what you have to watch out for is that the tool provides the necessary concepts to support the toolkit abstractions discussed above. If a tool is advertised to generically support testing every Web page there is, be cautious. This usually means that actions will be emulated coordinate based, sometimes ignorant of the type of elements. This will break sooner than you think.

• Object Names and Object Map: While DOM objects may have unique IDs or names, this isn't always the case. So the testing tool has to provide other means to identify an object. It can do so by considering a set of properties that - taken together - make up a unique identification. This again requires knowing the Web client framework built into the testing tool but allows for the development of robust tests.

     <span class="button" onClick="search();">
       <p>Search</p>
     </span>

  An HTML element without a single unique id attribute

  {tagName='SPAN' innerText='Search' }

  Generated attribute tuple identifying the element

  Using an object map that maps symbolic names (as used in the test script) to real names (as used in the Web application) also eases maintenance. If an object name changes in the application, it only requires one change in the central object map without touching the test cases.

• Scripting Language: Test cases should be written in a scripting language that provides common language features such as conditions, loops, subroutines, and so forth in addition to functions to access files, network, databases, etc. This creates sophisticated tests. Tools that only provide a list of statements for test cases are too limited to do anything useful except a few simple tests.

  As an example imagine creating a test for an online store. You probably want to test adding multiple items to the shopping cart. To implement that, a loop is required. Also, imagine that you don't want the test to add every item to the shopping cart but only those that cost less that $10. For this you'll need a conditional. Finally, you might want to check that the order has been written to the database correctly. For this you'll need a scripting language that offers a database API.

  To be able to implement such a test properly, you'll need a versatile scripting language for your test case.

Page 1 of 2   next page »