This content is reprinted from Real-World AJAX: Secrets of the Masters published by SYS-CON Books. To order the entire book now along with companion DVDs for the special pre-order price, click here for more information. Aimed at everyone from enterprise developers to self-taught scripters, Real-World AJAX: Secrets of the Masters is the perfect book for anyone who wants to start developing AJAX applications.

Building AJAX-Friendly Web Services
When developers first realize what an AJAX client can do, they are often especially excited about its potential in playing the role of the View or even the Controller in the Model-View-Controller (MVC) patterns of application development, with Web services providing the Model layer. If you're unfamiliar with the term, MVC is a paradigm used by software developers to design user interface code (the View) that is decoupled from the data objects it displays (the Model). Rather than letting the View code directly manipulate the Model's data, the MVC pattern introduces a third party known as the Controller. The Controller's job is to mediate between the View and Model, reacting to user interface input, updating data objects accordingly, and then updating the UI to reflect any changes in the data.

This same paradigm can be applied on a broader enterprise level across many systems. Service oriented architectures are designed to provide concise interfaces to business objects and their behaviors, exposing data to remote clients via HTTP and protocols built on it. AJAX strikes many as an ideal SOA client platform: a browser-based client provides an HTML renderer to display the View, a JavaScript environment to host Controller code, and an XMLHttpRequest to act as a communication mechanism for data exchange with a remote SOA-based Model.

In this chapter, we'll look at leveraging Web services in your AJAX applications. We'll look at strategies for formatting the payloads returned from Web services and at the many frameworks and libraries that are available that facilitate the development of AJAX applications. We'll also look at XML Remote Procedure Calls and messaging in full-blown SOAP-based Web services and techniques for managing their security and availability. First, however, we'll want to get one major stumbling block out of the way: AJAX's general inability to call services from other domains.

Note: Web Services, in upper case, refer specifically to SOAP-based exchanges that we'll discuss later. In this chapter, however, we're looking more at Web services, in lower case, which are generally understood to refer to the broader spectrum of any kind of service that can be accessed over HTTP.

Limitations on Cross-Domain AJAX
Regrettably, there are a number of security vulnerabilities serious enough that browser makers limit the HTTP requests made by the XMLHttpRequest object to the same domain from which the Java Script originates. So while you might have thought you could jump right in and write a light RSS feed reader in the corner of your blog to output content directly from your favorite news service, you will in fact probably have to HTTP to a CGI on your own server to proxy for you - that is, to retrieve and relay the RSS newsfeed back to you. As you can imagine, this is a matter of some controversy and because it impacts support for Web services in AJAX applications and frameworks so dramatically, it's worth spending a little time at the outset reviewing some of the potential security vulnerabilities that have caused browser makers to adopt this policy.

Perhaps the most obvious vulnerability is less a security issue than a performance issue: by calling a third-party service, the responsiveness and behavior of your interface can be made dependent on the availability and performance of another domain's servers. Your request for the other server's content will probably be asynchronous, but you'll still want to develop some additional error handling and perhaps a substitute for the content that your code may occasionally find itself unable to retrieve and leverage.

Conversely, if you syndicate content or otherwise advertise services from your own server, you may be setting yourself up to become a victim of resource theft, which occurs when other sites abuse your services by calling them beyond the level of service you originally intended. This vulnerability is, of course, already present in browser support for frames. But if you're exposing syndicated content or specific services, you'll want to consider some extra code to validate or throttle requests or at the very least keep an eye on your Web server's access log. Later in this chapter, we'll talk more about degrading gracefully around hung services and throttling client requests on the server side.

Another significant issue is Cross-Site Scripting (XSS), which occurs when an attacker manages to compromise the apparent origin of client-side scripting languages. This usually happens when some functionality on your page allows users to contribute their own content and you fail to escape any HTML or JavaScript in that content before re-presenting it as a part of your own page. The comment sections below blog postings are a common stage for this kind of attack. As with most of the other security vulnerabilities described here, AJAX doesn't make your site any more vulnerable to XSS - it's just something to keep in mind.

You may be able to digitally sign your JavaScript, indicating to the browser that your script can be trusted not to do anything malicious and that the restriction on what data the XMLHttpRequest object is allowed to retrieve can be waived. But the fact of the matter is that most browsers don't support cross-domain AJAX and are unlikely to support it any time soon. So if you want to use another domain's data on your Web site as part of an elaborate mashup or news portal, for instance, you'll probably be doing so via a proxy on your own server. We'll explore cross-domain proxying further in this chapter.

Now that we've clarified some of the reasons for limiting XMLHttpRequests to your own domain, we'll spend the rest of this chapter discussing various specifications for formatting your requests and responses. Many of these specifications are responses to the nature of HTTP itself, such as its statelessness or the content of its headers. So first, let's review the HTTP specification.

HTTP Is a Fundamental Transport
One of the most striking features of a richly interactive Web site built on AJAX is the page's resemblance to a desktop application in behavior and responsiveness. One might be inclined to suppose that such richness could only be achieved through ActiveX objects or a Flash plug-in that leverage proprietary mechanisms for data retrieval and display. But, as mentioned before, the key to these experiences is the asynchrony of the interface's behavior - that is, its ability to do multiple things at once, but over the same HTTP transport used by the browser in which it's running.

HTTP, the HyperText Transmission Protocol, is the familiar and ubiquitous dance of request and response behind the Web sites we surf everyday. Regardless of whether you're formatting the content of your request as a simple parameter in the URL's query string, as schema-validated XML, or enveloped in SOAP, you're still just using HTTP, which is mighty convenient given that HTTP is also one of the least likely protocols to be blocked at the firewall. Suffice it to say that if visitors can successfully retrieve your Web page, you can be assured of being able to continue to pass them data on that port and protocol, something you might not necessarily be able to assume with alternative channels.

Before looking at the structure of an HTTP exchange, the reader is encouraged to follow along by observing live requests and responses using tools like the Mozdev's LiveHTTPHeaders plug-in for Firefox at http://livehttpheaders.mozdev.org/ or Rex Swain's HTTP Viewer at www.rexswain.com/httpview.html. Fire one up and you'll see your client, or user agent, submitting a simple request message over a Transmission Control Protocol (TCP) connection:

GET /index.asmx HTTP/1.1
Host: www.myblog.com

GET is not the only request method the client could send, however. The client could also request only the header of a document, using the HEAD method. Or another important request method is POST, which you may recognize as the method you often specify in HTML forms. Using the POST method, the client transmits user data as a part of the request. There are other methods, but to all requests, the host should respond with something like:

HTTP/1.1 200 OK
Date: Tue, 21 May 2006 21:18:36 GMT
Server: Apache/1.3.27 (Unix)
Last-Modified: Tue, 12 Feb 2006 04:11:15 GMT
Content-Length: 54
Connection: close
Content-Type: text/html; charset=UTF-8

Following this response will be a blank line, followed again by the actual content of the requested page: the HTML, CSS, and JavaScript that the browser will use to render the page. Notice the first line of the host's response containing the HTTP status code (200 OK). There are many status codes that indicate everything from restricted resources to various server errors. A status code of 200 is what you'll be looking for most of the time and you'll notice many of the AJAX examples in this book check for a status code of 200 besides checking for the XMLHttpRequest's readyState of 4.

Note also the character encoding indicated in Content-Type. You'll want to make sure you're not getting JavaScript errors because of wrongly encoded content. Beyond that, there's not much more to say about HTTP because what's really important is the data following that first blank line - that is, what's in your payload and how you format it.

XML or JSON
One of the most appealing features of AJAX design patterns in Web development is the speed with which content can be delivered to the user. With JavaScript retrieving data directly from the server via the XMLHttpRequest object without the overhead of the surrounding HTML, the only real remaining bottleneck is the potential size of the XML payloads. So it's not surprising to find many AJAX developers wondering whether there are ways to streamline these payloads so there's as little syntactic overhead as possible.

There's been some progress with XML compression and Binary XML, but given that these payloads will ultimately be parsed by JavaScript running in the client, Douglas Crockford wondered if it wouldn't make sense to convey data in JavaScript to begin with, leaving it to developers of serverside languages to develop libraries to translate between the data formatted in JavaScript and the data formatted in XML or in other structures. And the development community has stepped up to the challenge and has shared libraries in a wide range of languages to do just that.

Crockford proposed JavaScript Object Notation (JSON), which uses a subset of the JavaScript syntax to support array and object literals. But JSON is just JavaScript and so lends itself to being more immediately intelligible to the client AJAX function. Before going into the data syntaxes behind JSON, let's quickly compare some data presented in XML, and then the same data presented in JSON:

Listing 6.1 XML

<department id="478" name="Human Resources">
   <employees>
     <employee id="1" lastname="Doe" />
     <employee id="2" lastname="Smith" />
     <employee id="3" lastname="Jones" />
   </employees>
</department>

Listing 6.2 JSON

{"department": {
   "id": "478",
   "name": "Human Resources",
   "employees": {
     "employee": [
       {"id": "1", "lastname": "Doe"},
       {"id": "2", "lastname": "Smith"},
       {"id": "3", "lastname": "Jones"}
     ]
   }
}}

The data behind this side-by-side comparison of JSON and XML is brief, but should suffice to illustrate the potential of much larger payloads of data represented in JSON to require substantially fewer bytes than their XML counterparts by virtue of conventions such as not requiring closing tags. Strip out any white space before transmitting to a client and the overall size of the data can be cut even more. One disadvantage of JSON is that it's not as human-readable as XML. XML's closing brackets, for instance, combined with the verbiage used in the elements, can go a long way in making it easier to understand the data and what it represents.

Again, JSON is just a subset of JavaScript, and specifically it's syntax for representing array and object literals. Array literals are specified using square brackets ([]) around a comma-delimited list of JavaScript values, which can be of any type:

var aPeople = ["John", "Mary", "Steve"]

And because arrays in JavaScript are not typed, you can mix types:

var aStuff = [37, "Thirty-Seven", null]

Object literals in JavaScript are used to store data in name-value pairs. The names and values are paired on either side of a colon (name : value), and then a comma-delimited list of these pairs is enclosed with two curly braces ({}):

var oBook = {
     "pages" : 467,
     "author" : "John Doe",
     "publisher" : "Perfect AC (USA)"
};

You can mix array and object literals by creating objects containing arrays and arrays of objects to get something like:

var aPeople = [
     {
       "firstname" : "John",
       "age" : 38
     },
     {
       "firstname" : "Mary",
       "age" : 27
     },
     {
       "firstname" : "Steve",
       "age" : 46
     }
];

This defines an array, aPeople, containing three objects, each having the property's firstname and age. Conversely, the values of each of these properties could themselves be literal arrays and so on, capturing some of the expandability of XML.

One security concern with JSON is that, because the JSON data is parsed using the eval() function, any additional JavaScript appended to that data can also be executed, posing a potentially serious security vulnerability to the browser. A safe way around this is to use Douglas Crockford's JSON parser, which will only recognize and process the arrays and object literals of JSON text:

var oMyObject = JSON.parse(sJSONData);

Crockford also provides a function stringify(), which will translate a JavaScript object into JSON:

var oPerson = new Object();
oPerson.firstname = "John";
oPerson.age = 36;
oPerson.favoriteColors = new Array("red", "green", "blue");

document.write(JSON.stringify(operson));

and this will output:

{"firstname":"John","age":36,"favoriteColors":["red,""green,"blue"]};

You can get the parse() and stringify() functions at www.json.org/json.js.

Another important resource provided at www.json.org is a collection of links to libraries in many languages that will encode and decode JSON data on the server side. ColdFusion developers, for instance, will be interested in Jehiah Czebotar's jsonencode.cfm User Defined Function at http://jehiah.com that they can include and then call in their code:

<cfinclude template="/udf/jsonEncode.cfm"> <!--- Include JSON UDF --->

<cfscript>
     oPerson = StructNew();
     oPerson.firstname = "John";
     oPerson.age = 36;
     oPerson.favoriteColors = arrayNew(1);
     oPerson.favoriteColors[1] = "red";
     oPerson.favoriteColors[2] = "blue";
     oPerson.favoriteColors[3] = "green";
</cfscript>
<cfoutput>#jsonEncode(oPerson)#</cfoutput>

which outputs:

{"age":36,"favoritecolors":["green"],"firstname":"John"}

and is ready to be requested by the client's AJAX engines. Similar libraries are also available to developers of C#, Java, Perl, and PHP.

Before leaving JSON, ColdFusion developers might also want to take a look at Rob Gonda's AjaxCFC, a ColdFusion framework designed to facilitate working with AJAX. What's interesting about Rob's approach is that his functions return pure JavaScript to the callback handler, which he finds vastly improves performance by eliminating the need to eval() the JSON object or parse the XML. AjaxCFC is based on the Java Open Source toolkit, Direct Web Remoting (DWR), which takes care of writing a lot of the low-level code required to get and manipulate the HTTP Request Object.

RSS as Web Service
RSS is a family of content feed formats that's particularly well suited for consumption by AJAX clients because its format is so consistent and widely used. Created by Dave Winter of UserLand Software, it's predominantly used to facilitate syndication of content from Web sites to let visitors to those Web sites track updates to the sites' content using an RSS aggregator.

But RSS is also often used in many organizations as a vehicle for tracking organizational or system events such as Help Desk tickets or application or system errors. And publicly, organizations are using RSS to replace e-mail distribution lists and e-mails to broadcast news and announcements. Like the other flavors of XML that we'll look at later, RSS is straight XML and can be translated, navigated, and otherwise manipulated by most server-side languages, and increasingly by browsers themselves.

First let's look at the structure of a simple RSS feed to familiarize ourselves with the main elements we'll either be translating or locating in JavaScript via the DOM. While earlier versions 0.9 and 1.0 are still used sometimes, we'll only look at the latest, 2.0.

Listing 6.3

<?xml version="1.0" encoding=" UTF-8" ?>
<rss version="2.0">
   <channel>
     <title>My Blog</title>
     <description>Random thoughts from my blog.</description>
     <link>http://myblog.com</link>
     <item>
       <title>My first post!</title>
       <link>http://myblog.com/myfirstpost.php</link>
       <author>me@myblog.com</author>
       <pubDate>Sat, 29 Apr 2006 14:20:12 GMT</pubDate>
       <description>
         I finally got a blog up and running - this is my first post!
       </description>
     </item>
   </channel>
</rss>

Following a standard XML declaration, the channel section identifies the title, description, and source of the feed. Following the channel section are any number of item sections that generally correspond to news headlines. Because RSS is intended to be a vehicle for the latest news, most RSS readers will only recognize the first 15 items in reverse chronological order.

Another syndication specification that's growing in popularity is Atom. Unlike RSS, Atom is an open standard and freely extensible. Among its distinguishing characteristics are strict specification of conventions like the distinction between escaped HTML and text content, the requirement of a globally unique ID, and the use of an XML namespace. Atom also has a few additional elements like the description, summary, and content elements. Contrast the following example of an Atom feed with the earlier example of an RSS feed.

Listing 6.4

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
   <title>My Blog</title>
   <subtitle>Random thoughts from my blog.</subtitle>
   <link href="http://myblog.com/"/>
   <updated>2006-29-13T18:30:02Z</updated>
   <author>
     <name>John Doe</name>
     <email>me@myblog.com</email>
   </author>
   <id>urn:uuid:60a76c80-d399-11d9-b91C-0003939e0af6</id>

   <entry>
     <title>My first post!</title>
     <link href="http://myblog.com/myfirstpost.php"/>
     <id>urn:uuid:1225c695-cfb8-4ebb-aaaa-80da344efa6a</id>
     <updated>2006-29-13T18:30:02Z</updated>
     <summary>I finally got a blog up and running - this is my first post!</summary>
   </entry>
</feed>

Again, because RSS and Atom are both just XML and can be parsed as such for content in your AJAX code, we'll back up now and consider in the next section the other various flavors of XML payloads you may find yourself working with.

This content is reprinted from Real-World AJAX: Secrets of the Masters published by SYS-CON Books. To order the entire book now along with companion DVDs, click here to order.