Testing
Exposing and Preventing Errors in AJAX Applications
Using a quality infrastructure
May. 16, 2007 07:15 PM
However, the methodologies for preventing security flaws haven't
changed. You should take a two-phased approach. The first phase is to
define a security policy that specifies how code should be written to
ensure security. A policy might specify things such as:
- Call a custom method you've defined called cleanInput() on any client input to the Web application to remove malicious input
- For Java applications, use the class Prepared-Statement for all DB queries to prevent SQL injection exploits
The second phase should involve functional validation that
the security policy is being followed. I discuss this below. With a
well-defined security policy, this two-phased approach should let you
build security into your application.
Application Source Code
Your application probably uses a
range of technologies and languages, including but not limited to CSS,
HTML, JSP, JSF, Java, ASP .NET, PHP, etc. All source code should be
validated using a set of best practices to ensure maintainability and
reliability. As with JavaScript, you should use an automated tool to
enforce the best practices that meet the goals and needs of your Web
application. The tool should have built-in best practice enforcement as
well as allow you to define your own internal best practices.
Application Server Output
Code that's delivered to the
browser from the server is often the result of an application server
processing server-side code (JSP, JSF, and ASP, for example) and
outputting HTML, JS, etc. As a result, it's often important to validate
the server output once it reaches the client, since there's no physical
representation of the "page" on the server. For AJAX applications, HTML
code often needs to contain certain elements (div or span tags, for
example) onto which AJAX libraries attach the user interface controls
that they define. Validating that such essential elements exist is
important in ensuring that developers don't inadvertently break page
components. This can be done by using an automated tool to define
additional best practices customized for an individual page or group of
pages, and enforcing them on the appropriate rendered pages. Enforcing
other best practices regarding well-formed and accessible HTML is also
important here.
Creating a Nightly Build/Test Infrastructure
Creating a nightly build/test infrastructure is the second major step
in building the overall quality infrastructure. At Parasoft we're
continually amazed at how many organizations don't do a nightly build.
Without a nightly build, how can you be sure that your current code
base even compiles? Or runs properly? Without a nightly build, errors
introduced into the application can exist for a long time before
they're detected. So, a nightly build is a crucial component of the
quality infrastructure. The nightly build must be easily repeatable
(meaning it can be run without user intervention), and must build the
full application.
Next, the infrastructure should run automated tests every night against
the application built by the nightly build. The tests should run
automatically without user intervention, and a report should be
generated and possibly e-mailed with the results of the run. The report
should categorize the results in an easy-to-read way, or else they'll
be ignored. But the infrastructure isn't enough. There must be a
culture in the development group where failures in the nightly tests
aren't allowed to remain. Failures must be addressed as soon as they
appear. Otherwise you lose the benefit of automated tests.
This is where the real power in the quality infrastructure lies. The
development group should adopt a practice that, as each new feature is
added, one or more automated tests that validate the feature are added
to the test suite. In addition, for each bug that's fixed, one or more
automated tests should be created that find the error being fixed.
Personally, some of my nightly tests are JUnit-based. I've made it a
practice to create a JUnit test that catches the bug I'm fixing before
I actually fix it. Then I know that my JUnit test reproduces the
problem, and by running it I can know when I actually do fix the
problem. This speeds up my development AND adds another test to my test
suite. By adopting these practices, you'll develop a large array of
tests in the long run. If they all succeed, you can be reasonably sure
that your application has no major flaws. If you have to put out an
emergency build and you have no time for testing, you can have a
reasonable amount of confidence that the application is stable and
works correctly if the automated tests succeed. This is especially true
in the world of Web applications, where features are bled into the
applications constantly.
Functional Tests for AJAX Applications
Since automated nightly tests are so important, let's focus on the
details of what should be functionally tested in an AJAX application.
Back-end Web Services
Many AJAX applications depend on Web
Services. These may be services that your company owns and manages, or
they may be services exposed by your partners. For Web Services owned
by your company, you should use an automated Web Services testing
solution to validate that the Web Services provide the functionality
they're supposed to, and that they handle both expected and unexpected
inputs. Your Web application depends on these Web Services providing
the expected data, so you need to ensure that they do. Whether you own
the Web Services or you use services exposed by someone else, you also
need to validate that your Web application correctly handles unexpected
responses from the Web Services (such as the service being
inaccessible, the service returning malformed or unexpected data,
etc.). This way you can be certain that your Web application will
provide an appropriate response under unexpected conditions.
Asynchronous Message Content
When an error occurs with the
page-update mechanism of an AJAX application, it's typically in one of
two places. The first place is in the contents of the asynchronous
messages used to communicate with the server. The second place can be
in the logic of the JavaScript engine that processes the asynchronous
messages and that updates the user interface. So it's important to
validate that the server returns expected content for the asynchronous
messages. Regression tests should be set up to verify that the server
returns the correct responses for both expected and unexpected inputs.
If there's a problem in the Web application, and these tests are
succeeding, then the problem must be somewhere in the client-side
JavaScript engine.
Client-Side JavaScript Engine
Validating the functionality
of a client-side JavaScript engine may be the most difficult aspect of
ensuring quality in an AJAX application. This is why it's important to
use both best practice enforcement and functional testing to attack the
problem from multiple angles. Functionally validating the JavaScript
engine is done by constructing functional tests for each user interface
control in your Web application. You should use an automated tool that
lets you record a user's actions as he interacts with a browser and set
up validations that ensure the control correctly responds to the user's
actions. Since what the user sees in a Web application is directly tied
to the browser's DOM, and since AJAX applications rely on heavy
manipulation of the DOM, the automated tool should be able to perform
validations at the level of the browser DOM. Tools that emulate a
browser, rather than actually using a real browser for testing, are
probably not going to be able to accurately test the application. The
tests should be component-level tests of individual page components,
rather than functional scenarios of the Web application as a whole.
When these tests are combined with accurate regression tests on
asynchronous message content, you can have confidence that your AJAX
components are working properly.
End-to-End Functional Scenarios
Automated functional tests
of the scenarios that describe your Web application users' actions are
crucial to ensuring that your Web application is providing its critical
business functions. This is where you validate that all aspects of the
complex AJAX application are working together properly. You should
identify what the important scenarios are, and then use an automated
tool that can record and replay the scenarios easily. The tool should
let you validate that the user interface updates properly, as well as
test that the user's actions can actually be done. As with validating
the JavaScript engine, the user interface validations should be done at
the level of the browser DOM. In addition, since it's probably
important for your Web application to support multiple kinds of
browsers, the test tool should let you define the tests once, yet run
them against the different browsers and different browser versions that
your application needs to support.
Functional Security Tests
Functional security tests ensure
that the security policy that's been put in place actually works and is
being followed. These tests should support, rather than replace, the
policy. The tests should send various kinds of unexpected inputs to the
Web application to validate that it responds appropriately to hacker
attacks (such as cross-site scripting and SQL injections). Server
responses should be validated to ensure that the application filters
out the inappropriate inputs and doesn't reveal information that would
help a would-be hacker with further attacks.
Conclusion
Creating a quality infrastructure will
let you deliver your applications quickly, but with confidence that
they are reliable and robust. This is especially true of AJAX
applications. Creating the infrastructure may require a significant
investment of time and resources upfront; however, experience with my
own development and with that of clients has shown that the gains from
finding and preventing errors early using a quality infrastructure are
many times greater than the initial cost.
About Nathan JakubiakNathan Jakubiak is a software engineer at Parasoft. He currently manages the development of Parasoft WebKing and is the lead developer of Parasoft's AJAX testing solution. Nathan has extensive experience developing testing tools for web-based technologies and has a number of pending patents in the area of automated regression testing.