Combine, minify and compress JavaScript files to load ASP.NET pages faster

18 Jan 2009

Websites are getting more interactive these days and most of this interactivity comes from JavaScript. People are using different JavaScript libraries and frameworks to make their websites more interactive and user friendly.

What is the problem?

After a while you end up referencing to more than a few JavaScript files in your page and that causes a problem! Your pages will load slower; this happens for two reasons:

  • You have used too much JavaScript. Total size of JavaScript files in some websites may reach more than 500KB and that's a lot, especially for users with slow connections. The best solution to this problem is to use gzip to compress JavaScript files as we do later in this article.
  • JavaScript files are loaded one by one after each other by the browser. As you know there is a response time for each request that varies depending on your connection speed and your distance from the server. If you have many JavaScript files in your page these times are added together and cause a big delay when loading a page. I have tried to show this using Firebug (A very popular and necessary FireFox extension for web developers) in the following screenshot.
    This is not the case for CSS or image files .If you reference multiple CSS files or images in your page they are loaded together by the browser.
    The best solution to this problem is to combine JavaScript files into one big file to remove the delay caused when loading multiple JavaScript files one by one.

Firebug shows networl activity for multiple JavaScript files loading

What can we do?

The answer is simple! Combine the files and gzip the response. Actually we are going to take few extra steps to create something more useful and reusable. But how we are going to do that?

The easiest way to do this is to use an HTTP Handler that combines JavaScript files and compresses the response. So instead of referencing multiple JavaScript files in our pages we reference this handler like this:

<script type="text/javascript" src="ScriptCombiner.axd?s=Site_Scripts&v=1"></script>

ScriptCombiner.axd is our HTTP Handler here. Two parameters are passed to the handler, s and v. s stands for set name and v stands for version. When you make a change in your scripts you need to increase the version so browsers will load the new version not the cached one.
Set Name indicates list of JavaScript files that are processed by this handler. We save the list in a text file inside App_Data folder. The file contains relative paths to JavaScript files that need to be combined. Here is what Site_Scripts.txt file content should look like:

~/Scripts/ScriptFile1.js  
~/Scripts/ScriptFile2.js

Most of work is done inside the HTTP Handler; I try to summarize what happens in the handler:

  • Load the list of paths JavaScript files from the text files specified by the set name.
  • Read the content of each file and combine them into one big string that contains all the JavaScript.
  • Call Minifier to remove comments and white spaces from the combined JavaScript string.
  • Use gzip to compress the result into a smaller size.
  • Cache the result for later references so we don't have to do all the processing for each request.

We also take few extra steps to make referencing this handler easier. Source code for this article can be accessed from github.com/atashbahar/script-combiner.

Here is a screenshot of the website opened in solution explorer so you get a better idea of how file are organized in the sample website.

Sample website opened inside solution explorer

The implementation

I have got most of the code for the handler from Omar Al Zabir article named "HTTP Handler to Combine Multiple Files, Cache and Deliver Compressed Output for Faster Page Load" so most of the credit goes to him.

HTTP Handler contains a few helper methods that their purpose is very clear. I try to describe each shortly.

CanGZip method checks if the browser can support gzip and returns true in that case.

private bool CanGZip(HttpRequest request)
{
    string acceptEncoding = request.Headers["Accept-Encoding"];
    if (!string.IsNullOrEmpty(acceptEncoding) &&
         (acceptEncoding.Contains("gzip") || acceptEncoding.Contains("deflate")))
        return true;
    return false;
}

WriteBytes writes the combined and compresses bytes to the response output. We need to set different content type for response based on if we support gzip or not. The other important part of this method sets the caching policy for the response which tells the browser to cache the response for CACHE_DURATION amount of time.

private void WriteBytes(byte[] bytes, bool isCompressed)
{
    HttpResponse response = context.Response;

    response.AppendHeader("Content-Length", bytes.Length.ToString());
    response.ContentType = "application/x-javascript";
    if (isCompressed)
        response.AppendHeader("Content-Encoding", "gzip");
    else
        response.AppendHeader("Content-Encoding", "utf-8");

    context.Response.Cache.SetCacheability(HttpCacheability.Public);
    context.Response.Cache.SetExpires(DateTime.Now.Add(CACHE_DURATION));
    context.Response.Cache.SetMaxAge(CACHE_DURATION);

    response.ContentEncoding = Encoding.Unicode;
    response.OutputStream.Write(bytes, 0, bytes.Length);
    response.Flush();
}

WriteFromCache checks if we have the combined script cached in memory, if so we write it to response and return true.

private bool WriteFromCache(string setName, string version, bool isCompressed)
{
    byte[] responseBytes = context.Cache[GetCacheKey(setName, version, isCompressed)] as byte[];

    if (responseBytes == null || responseBytes.Length == 0)
        return false;

    this.WriteBytes(responseBytes, isCompressed);
    return true;
}

But most of the work is done inside ProcessRequest method. First we take a look at the method and I will try to explain important parts.

public void ProcessRequest(HttpContext context)
{
    this.context = context;
    HttpRequest request = context.Request;        

    // Read setName, version from query string
    string setName = request["s"] ?? string.Empty;
    string version = request["v"] ?? string.Empty;

    // Decide if browser supports compressed response
    bool isCompressed = this.CanGZip(context.Request);

    // If the set has already been cached, write the response directly from
    // cache. Otherwise generate the response and cache it
    if (!this.WriteFromCache(setName, version, isCompressed))
    {
        using (MemoryStream memoryStream = new MemoryStream(8092))
        {
            // Decide regular stream or gzip stream based on whether the response can be compressed or not
            using (Stream writer = isCompressed ? (Stream)(new ICSharpCode.SharpZipLib.GZip.GZipOutputStream(memoryStream)) : memoryStream)                
            {
                // Read the files into one big string
                StringBuilder allScripts = new StringBuilder();
                foreach (string fileName in GetScriptFileNames(setName))
                    allScripts.Append(File.ReadAllText(context.Server.MapPath(fileName)));

                // Minify the combined script files and remove comments and white spaces
                var minifier = new JavaScriptMinifier();
                string minified = minifier.Minify(allScripts.ToString());

                // Send minfied string to output stream
                byte[] bts = Encoding.UTF8.GetBytes(minified);
                writer.Write(bts, 0, bts.Length);
            }

            // Cache the combined response so that it can be directly written
            // in subsequent calls 
            byte[] responseBytes = memoryStream.ToArray();
            context.Cache.Insert(GetCacheKey(setName, version, isCompressed),
                responseBytes, null, System.Web.Caching.Cache.NoAbsoluteExpiration,
                CACHE_DURATION);

            // Generate the response
            this.WriteBytes(responseBytes, isCompressed);
        }
    }
}

this.WriteFromCache(setName, version, isCompressed) returns true if we have the response cached in memory. It actually writes the data from cache to the response and returns true so we don't need to take any extra steps.

MemoryStream is used to hold compressed bytes in the memory. In case that browser does not support gzip no compression is done over bytes.

We use SharpZipLib to do the compression. This free library does gzip compression slightly better than .NET. You can easily use .NET compression by replacing line 20 with the following code:

using (Stream writer = isCompressed ?  (Stream)(new GZipStream(memoryStream, CompressionMode.Compress)) : memoryStream)

GetScriptFileNames is a static method that returns a string array of JavaScript file paths in the set name. We will use this method for another purpose that we discuss it later.

JavaScriptMinifier class holds the code for Douglas Crockford JavaScript Minfier. Minifer removes comments and white spaces from JavaScript and results in a smaller size. You can download the latest version from http://www.crockford.com/javascript/jsmin.html.

After we wrote minified and compressed scripts to MemoryStream we insert them into cache so we don't need to take all these steps for each request. And finally we write the compressed bytes to the response.

Make it a little better

We can take one extra step to make the whole process a little better an easier to use. We can add a static method to our handler that generates JavaScript reference tag. We can make this even better by doing different actions when application is running in debug or release mode. In debug mode we usually don't want to combine our JavaScript files to track error easier, so we output reference to original files and generate all JavaScript references as if we have included in our page manually.

public static string GetScriptTags(string setName, int version)
{
    string result = null;
#if (DEBUG)            
        foreach (string fileName in GetScriptFileNames(setName))
        {
            result += String.Format("\n<script type=\"text/javascript\" src=\"{0}?v={1}\"></script>", VirtualPathUtility.ToAbsolute(fileName), version);
        }
#else
    result += String.Format("<script type=\"text/javascript\" src=\"ScriptCombiner.axd?s={0}&v={1}\"></script>", setName, version);
#endif
    return result;
}

GetScriptFileNames as described before returns an array of file names for a specified set.

// private helper method that return an array of file names inside the text file stored in App_Data folder
private static string[] GetScriptFileNames(string setName)
{
    var scripts = new System.Collections.Generic.List<string>();
    string setPath = HttpContext.Current.Server.MapPath(String.Format("~/App_Data/{0}.txt", setName));
    using (var setDefinition = File.OpenText(setPath))
    {
        string fileName = null;
        while (setDefinition.Peek() >= 0)
        {
            fileName = setDefinition.ReadLine();
            if (!String.IsNullOrEmpty(fileName))
                scripts.Add(fileName);
        }
    }
    return scripts.ToArray();
}

<%= ScriptCombiner.GetScriptTags("Site_Scripts", 1) %>

So whenever we want to reference a handler in an ASP.NET page we only need to add the following tag inside the head section of our page:

There is a final step you need to take before making all this work. You need to add HTTP Handler to Web.config of your website. To do so make the following changes to your web.config file:

<configuration>
    <system.web>
		<httpHandlers>
			<add verb="POST,GET" path="ScriptCombiner.axd" type="ScriptCombiner, App_Code"/>
		</httpHandlers>
	</system.web>
	<!-- IIS 7.0 only -->
	<system.webServer>
		<handlers>
			<add name="ScriptCombiner" verb="POST,GET" path="ScriptCombiner.axd" preCondition="integratedMode" type="ScriptCombiner, App_Code"/>
		</handlers>
	</system.webServer>
</configuration>

Conclusion

If you search the Internet you will find many similar methods. I have tried to combine the best parts of them to make something more useful. I hope you have enjoyed it.