Real-World AJAX Book Preview: Web Server-Based Compression

May. 6, 2007 11:00 AM

This content is reprinted from Real-World AJAX: Secrets of the Masters published by SYS-CON Books. To order the entire book now along with companion DVDs for the special pre-order price, click here for more information. Aimed at everyone from enterprise developers to self-taught scripters, Real-World AJAX: Secrets of the Masters is the perfect book for anyone who wants to start developing AJAX applications.

Web Server-Based Compression
One way of speeding up transfer times of resources between the server and the browser is to reduce the size of the resource being transferred. Practically all modern browsers can receive and decompress compressed content via HTTP. Using one of the HTTP traffic-monitoring tools mentioned above, you can see that browsers send out a header, Accept-Encoding=gzip,deflate, that tells the server that it can get compressed content. For these browsers, you can configure your Web server to compress content on-the-fly for dynamic content and cache compressed versions for static content. For Apache 2 you can use the mod_deflate module to add the capability for automatic compression. The mod_deflate module is included in the distribution of Apache 2, so compiling it into your Apache 2 installation is just a matter of specifying the right configuration flag:

./configure --enable-modules=all --enable-mods-shared=all --enable-deflate

To configure your server to use mod_deflate, you need to enable it first. If you've installed Apache in the default path, you'll have to add the following to your httpd.conf file:

LoadModule deflate_module /usr/lib/apache2/modules/mod_deflate.so

To turn on compression for the content types that are of interest to AJAX, add the following line to your top-level <Location> or <Directory> element:

AddOutputFilterByType DEFLATE text/html text/plain text/xml text/JavaScript text/
css application/x-JavaScript

There are a number of old browsers that advertise that they can handle compressed content but really can't. To address this issue, we add some additional configuration lines to handle these browsers:

Netscape 4.x has some problems...BrowserMatch ^Mozilla/4 gzip-only-text/html
Netscape 4.06-4.08 have some more problems BrowserMatch ^Mozilla/4\.0[678] no-gzip
MSIE masquerades as Netscape, but it is fine BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

To log the compression ratio, add the following to your custom log configuration:

DeflateFilterNote Ratio deflate_ratio
LogFormat '"%r" (%{deflate_ratio}n%%)' deflate
CustomLog /var/log/apache2/deflate.log deflate

For text files, the compression ratio can be as much as 70%-80%. The default compression algorithm for mod_deflate is tuned for optimal server performance, but you can increase or decrease the compression level, from one to nine, using the DeflateCompressionLevel directive:

# maximum compression
DeflateCompressionLevel 9

IIS is also capable of deflating content. To enable HTTP compression in IIS before 6.0, use the IIS Manager and check the boxes for static and dynamic content.

To enable compression in IIS 6.0, navigate to the IIS snap-in, right-click on the Web Sites node, and select Properties, then click on the Service tab. Check the boxes for static and dynamic content.

The downside to using compression is that it consumes CPU resources for each download, both at the server and the client, as the content must be compressed and uncompressed. Typically the tradeoff favors compression, especially in the case of traffic over the public Internet, where data transfer times are likely to be comparatively longer than data compression and decompression times. The only way to tell for sure is to log the performance of the application over time using one of the HTTP traffic monitoring tools such as TamperData.

Compacting JavaScript
Well-formatted and commented JavaScript and CSS are pretty fat. All that white space and comment information makes the code easier to read, but it adds bytes that aren't useful to the computer, parsing and executing. The same goes for those long descriptive variable and function names - great for humans, unnecessary for computers. Fortunately there are ways of crunching the size of this code by eliminating the unnecessary material. A word of warning: this is one of those places where optimization gets in the way of supportability. Don't try to write and debug code that trims the readability fat. Instead, write and debug code that's as readable as you can, then compact it before deploying it into production. Note too that some of the more aggressive compression approaches can actually introduce defects into your code. You should always regression-test your code after transformation with a tool like JSUnit to make sure it still behaves the same (www.jsunit.net/).

There are a number of tools that let you compact or compress your JavaScript. One of the least intrusive is JSMin by Douglas Crockford (www.crockford.com/JavaScript/jsmin.html). JSMin essentially performs the simple transformation we outlined above - it eliminates comments and extra white space, but doesn't change any of the identifiers. JSMin's code compression is modest but safe. As an example, our XHRPool looks as follows when run through JSMin:

The Dojo toolkit's ShrinkSafe is a tool that compresses JavaScript more aggressively, without changing the public variables or APIs of the code (http://alex.dojotoolkit.org/shrinksafe/). The secret here is Rhino, a JavaScript interpreter written in Java (www.mozilla.org/rhino/). ShrinkSafe loads the JavaScript code into Rhino and transforms variable names and other long identifiers using Rhino's internal representation. Compression with this tool is better than with JSMin, but since the transformation is more substantial, you must test the resulting code, as mentioned before. You can see our XHRPool with ShrinkSafe compression below:

A different approach is taken by Dean Edwards' Packer (http://dean.edwards.name/packer/). Packer transforms the JavaScript into a text string, which lets long JavaScript keywords, like “function," be replaced by shorter text strings. This can save quite a bit of space in large JavaScript files. When loaded into the browser, the keywords are replaced using a regular expression before the JavaScript string is evaluated, i.e.. using eval(). One gotcha with this tool is that all of your lines of JavaScript code must end with a semicolon, otherwise the resulting code will be corrupt.

Finally, the most aggressive JavaScript compactor is Memtronic's JavaScript/HTML Cruncher-Compressor (http://hometown.aol.de/_ht_a/memtronic/). This compactor/compressor also suffers from Packer's semicolon issue. The resulting output is barely decipherable to the point of being obfuscated.

A tool for optimizing CSS is CSSTidy (http://csstidy.sourceforge.net/). It trims unneeded white space and semicolons, removes comments, and merges properties where possible. You can use an online version of this tool at CSSClean (www.cleancss.com/).

Which compressor is right for you? I'd suggest using JSMin or ShrinkSafe, since they seem to be the most reliable and work with the most browsers, as well as CSSTidy. When combined with server-side compression, the space savings that these compactors achieve should be adequate for most applications.

Combining Resources
One additional limit on network performance in browsers is that they can make no more than two simultaneously persistent connections to the same host. This is part of the HTTP protocol specification (RFC 2616) and so is likely to continue to be the default behavior for browsers. It's possible for users to tweak this limit, but you shouldn't count on that being the case when designing your application. This limit applies to any kind of content, whether JavaScript, CSS, images, or XHTML.

Since one common approach to increasing the perceived performance of an AJAX application is to preload as much material - images, XHTML, etc. - as possible, loading dozens of images can chew up quite a bit of time as they queue up to be loaded two at a time. One approach to overcoming this limit is to load different resources from different hosts. For example, load all of your images for www.domain.com from images.domain.com, which gives you four effective connections for loading resources. You can extend this idea even further to JavaScript.domain.com, css.domain.com, or even further to images1.domain.com, images2.domain.com, and so forth.

This sort of optimization can introduce a lot of complexity into your application, as you manage DNS and Web server configuration and the rewriting, automatic or otherwise, of your HTML. Also, the overhead associated with lots of connections isn't inconsequential.

Rather than allowing more connections, a better approach is to require fewer. One way to do this is by combining as many resources as possible. The simplest and most obvious optimization is to concatenate all of your JavaScript libraries into one file and all of your CSS files into another. Combine this consolidation with the previously discussed crunching and compressing, and you can save substantially on startup times.

Combining images can save even more time, since Web applications tend to contain far more images than CSS or JavaScript files. One way of combining images is by grouping images that appear together into a single image accessed by a client-side imagemap.

<html>
<head>
<title>Imagemap Combination</title>
</head>
<body>
<map name="toolbar">
<area shape="rect" coords="0,0,31,31" href="JavaScript:alert('new')">
<area shape="rect" coords="32,0,63,31" href="JavaScript:alert('open')">
<area shape="rect" coords="64,0,95,31" href="JavaScript:alert('save')">
<area shape="rect" coords="96,0,127,31" href="JavaScript:alert('print')">
</map>
<h1>Toolbar</h1>
<img src="map.png" usemap="#toolbar" border="0">
</body>
</html>

Our example uses four 32x32 pixel icons, glued together using Photoshop. There are some automated ways of stitching images together.

Another, more flexible, and efficient way of combining images into a single download is to splice them together into a big sheet of images and then use CSS to display only single image portions of the sheet. We do this by creating a class that positions the sheet at the top-left corner of the target image, crops the height and width of the target image's size, and hides the overflow.

<html>
<head>
<title>Image Combination</title>
<style type="text/css">
.NewIcon {
    background-image:url(“map.png");
    background-position:0px 0px;
    background-repeat:no-repeat;
    width:32px; height:32px;
    overflow:hidden;
}
.OpenIcon {
    background-image:url(“map.png");
    background-position:-32px 0px;
    background-repeat:no-repeat;
    width:32px; height:32px;
    overflow:hidden;
}
.SaveIcon {
    background-image:url(“map.png");
    background-position:-64px 0px;
    background-repeat:no-repeat;
    width:32px; height:32px;
    overflow:hidden;
}
.PrintIcon {
    background-image:url(“map.png");
    background-position:-96px 0px;
    background-repeat:no-repeat;
    width:32px; height:32px;
    overflow:hidden;
}
</style>
</head>
<body>
<h1>Four Separate Icons</h1>
<div class="NewIcon"></div>
<div class="OpenIcon"></div>
<div class="SaveIcon"></div>
<div class="PrintIcon"></div>
</body>
</html>

We can extend this technique to save a little bit of bandwidth by applying some additional styles to the icons in the example above, making them appear as disabled or “greyed out" versions. We do this by giving it an opacity of 0.35.

[snip]
.NewIcon, .NewIconDisabled {
    background-image:url(“map.png");
    background-position:0px 0px;
    background-repeat:no-repeat;
    width:32px; height:32px;
    overflow:hidden;
}
.NewIconDisabled {
    opacity:.35;filter:alpha(opacity=35);
}
[snip]

<h1>Active and Disabled Icons</h1>
<div class="NewIcon"></div>
<div class="NewIconDisabled"></div>
<div class="OpenIcon"></div>
<div class="OpenIconDisabled"></div>
<div class="SaveIcon"></div>
<div class="SaveIconDisabled"></div>
<div class="PrintIcon"></div>
<div class="PrintIconDisabled"></div>
[snip]

Using the techniques above, you can reduce the number of HTTP connections your browser needs to make to just three, one for your JavaScript, another for your CSS, and a third for your images. Unfortunately, there are some quirks in particular browsers that cause them to reload resources. The way to deal with this is through caching.

Tweaking the Cache: The Art of Not Downloading At All
The best way to speed up the downloading of server-side resources is not to download them in the first place, but pull them out of a cache instead. Browsers already have a cache that maps URLs to content stored locally on disk.

How does this browser cache work? To understand it we have to look at the HTTP 1.1 request response cycle. (We'll gloss over some of the nitty-gritty details for clarity.) When a browser asks for a static resource, such as HTML, JavaScript, CSS, or an image, it sends a GET request to the server. The server sends back the requested resource along with a Last-Modified header telling the browser the last time the resource was modified. The resource and the modification timestamp are both stored in the browser's cache. The next time the resource is needed, say through another page load, the browser makes another GET request but sends along a header based on the modification timestamp. If the resource on the server has been modified more recently, the server sends back the resource with the new timestamp in the Last-Modified header. If the resource hasn't changed, the server sends back a response with a 304 status code, indicating to the browser that the resource hasn't changed. In that case, the browser uses the resource from its cache.

There are only two things wrong with this behavior. First, the browser has some heuristics that determine how long it considers a resource to be fresh. Second, if the resource isn't fresh, there's a round-trip for the validation even if a valid resource is in the cache. The heuristics vary from browser to browser, but the timeout is usually fairly short. (See RFC 2616 for detail: www.w3.org/Protocols/rfc2616/rfc2616-sec13.html.) If your application is like most, your images, CSS, and JavaScript probably don't change that often. The HTTP protocol provides headers for controlling how the browser caches information by sending a max-age directive. The max-age directive explicitly tells the browser how long it can consider a resource fresh.

In Apache, you can use the mod_expirers module to set the max-age for your resources. To set an expiration time of six months on your resources, add the following lines to your Apache configuration file:

ExpiresActive On
ExpiresDefault “access plus 6 months"

You can control the expiration by type as follows:

ExpiresByType text/JavaScript “access plus 6 months"
ExpiresByType application/x-JavaScript “access plus 6 months"
ExpiresByType text/css “access plus 12 months"
ExpiresByType image/png “access plus 12 months"

In IIS, open the Internet Service Manager, open the property dialog for the file or resource on which you want to set an expiration, and click on the HTTP Headers tab.

Check “Enable Content Expiration," select the “Expire after," and enter the expiration time span you want. On IIS, this setting will send both the Cache-Control header and the Expiration header, an alternate way of controlling the cache.

If you do end up changing your CSS or JavaScript, you can force a reload simply by adding a version number to your filenames:

When you make a change, simply increment the version number and the browsers will load since they constitute a new URL.

There are some gotchas, nonetheless, with IE6 and caching. Consider the following code:

function insertImages() {
var elem = document.getElementById(“test1");
elem.innerHTML += '<img src="hugeImage.png"/><img src="hugeImage.png"/>';
}

If you clear your browser cache and run this in IE6 via an event, you'll see that the browser makes two requests for the same image. The same thing happens when you construct image DOM nodes and append them via document.appendChild() or if you introduce the images via a CSS background image (which will be rather important when we look at combining images later on). One workaround for this is to pre-cache the images like so:

Now when the code above is executed, the image isn't loaded at all since it's already in the browser cache.

Conclusion
Performance tuning AJAX applications can introduce complexity into your code and architecture so make sure that performance is really an issue before you start. Once you're convinced you need to tune, make sure you can measure performance so you know where the bottlenecks are and whether your tuning measures are effective.

The tools and techniques covered in this chapter will help you measure performance, detect resource leaks, and improve many other aspects of AJAX application performance with a minimum of effort. Profiling tools such as Venkman let you identify hotspots and inefficiencies in your code. Memory profilers such as Drip help you identify and eliminate memory leaks. Programming techniques like object pooling can reduce the memory footprint in object-heavy applications.

Code compression tools such as ShrinkSafe will reduce the size of your JavaScript, CSS, and XHTML to load quicker as will HTTP compression through Apache and IIS Web server modules. Using JSON, rather than XML, can also reduce the size of your response payloads.

Combining scripts, CSS files, and even images will overcome the two connection limitations, speeding the download of your application. Setting the cache control and other headers for static resources on the server can prevent connections from happening at all by preventing unnecessary image downloads during DOM insertions. Tools like TamperData will help in this by letting you analyze the request and response behavior of your application.

Keep your eye out for new tools, especially for Internet Explorer. In most cases we're making do with tools designed for traditional, pre-AJAX applications, so the demand for new tools is strong and growing. Also make sure to inform yourself about the performance characteristics and features of new browser versions. As of this writing, the paint wasn't dry on Internet Explorer 7, so it's not quite clear if it will fix any of the performance or memory leak problems of its predecessor.

When all else fails, optimize your application so that it appears to perform well. Tools like Web Page Analyzer will help you diagnose issues that may be keeping your pages from loading quickly. Other techniques like using progress bars and incremental updates can give the user the experience of a responsive application, even if the underlying operations are slow.

Finally, follow good coding practices. Well-written code is less likely to exhibit performance problems than poorly written code, and is much easier to optimize than awkward, inflexible, poorly designed code.

This content is reprinted from Real-World AJAX: Secrets of the Masters published by SYS-CON Books. To order the entire book now along with companion DVDs, click here to order.

About Dietrich Kappe
Dietrich Kappe is a co-founder and the CTO of Pathfinder Development, a firm that combines User Experience Design and Agile to speed software product development. He published one of the first 100 public websites and launched one of the first Java Servlet-based web applications. He has been a software engineer for two decades, a frequent open source contributor, and has developed applications for the Media, Financial Services, Insurance and Healthcare industries.