Solving client resources caching problem
Note: Special thanks to Sergey and Yuri for implementation example
Let's get back to the old client caching problem.
Consider the following scenario:
We have a web based application that contains static client resources (*.js, *.css, *.htc etc). We would like to exploit downstream cache, and be absolutely sure that the client receives up to date file version. Possible solutions are:
1. Play around with cache expiration header. Yuri had just posted the right way to do it. In case that the client's request had reached the server, the resource expiration date will be checked, and the updated resource or 304 (not changed) will be served. The main disadvantage of this method is a need for a client to make a roundtrip on each access to the resource. Even if we completely control all clients and make them ask for a new version of file on every visit (Settings of IE), we will still have to trust that all proxies between our client and IIS are playing nice and are not just checking against their own local cache. Even if we did manage to ensure all this, we will still get roundtrip on every visit to everything! Having not-so-wide connection, this can result in flickering and unnecessary server load.
2. Change resource URL with every update. Advantage is clear. It is absolutely bullet proof. You can use the most aggressive caching mechanism and still be sure resource is OK. Browser will have to go the whole way because the item with a new URL is not found anywhere. Unfortunately, disadvantage is also clear: It's a hell to support such a deployment scenario.
3. Whidbey client resource caching approach.
4. Any other ideas?
Whidbey is not here yet for a while, and we have variety of ASP.Net systems out there in production. What can we do today (that is better than instruct our clients to clear browser cache each time the problem occurs)?
We would like to change URL but don't want to deploy changes differently. Let's steal some Whidbey ideas. We will construct some mechanism that automatically modifies resource URL on each update. On the server side we'll use custom HTTP handler to map modified URL to real file.
Let's start with URL creation. It will be constructed as <original_file_name>_<version>_<content_type>_InfraScriptCache.js. First of all we will use version indicator from Web.config file to automatically modify URL. Pay attention to resource extension. It appears that majority of browsers don't trust unknown file extensions, and if the browser doesn't trust something, it will travel to server to check and to be on the safe side. To prevent it we let the browser treat all resources as js files and incorporate real content type into resource URL. Extending URL with "_InfraScriptCache.js" is our way to let custom handler know it should process this request. We don't want occasional js files receive special treatment.
Now let's look at HTTP handler. It is really simple. The handler has to do just a couple of things:
1. Check "If-Modified-Since" header. If it exists in request, browser has the right version and is checking for updates. There is no change in version number, no need to proceed, just to serve 304 to client.
2. If "If-Modified-Since" is not here, browser is looking for updated/new file for the first time. Now we read the file, optionally in-memory cache it, set expiration date to a year from now (some browsers will disregard longer expiration period), and serve file.
Having both ends, all we have to do is setup Http handler on IIS (one time installation), and update version indicator in Web.config on each resource update.
Appendix A.
Registration Custom Control
using System;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.ComponentModel;
using System.Collections;
namespace WebTest
{
public class RegistrationControl : Control
{
private WebResourceCollection m_Resources = new WebResourceCollection();
public WebResourceCollection Resources
{
get
{
return this.m_Resources;
}
}
protected override void OnPreRender(EventArgs e)
{
foreach (WebResourceDescription res in this.m_Resources)
{
// Use original file name to identify resource on page
if (this.Page.IsClientScriptBlockRegistered(res.Name))
{
this.Page.RegisterClientScriptBlock(res.Name,
GetWebResourceTag(GetWebResourceURL(res.Name, res.ContentType), res.ContentType));
}
}
}
protected override void Render(HtmlTextWriter output)
{
// Nothing to render
}
public static string GetWebResourceTag(string resourceURL, ContentType contentType)
{
switch (contentType)
{
case ContentType.css:
return String.Concat(@"<LINK href="", resourceURL, @"" type=""text/css"" rel=""stylesheet"">");
case ContentType.js:
return String.Concat(@"<script language=""jscript"" src="", resourceURL, @"" ></script>");
case ContentType.vbs:
return String.Concat(@"<script language=""vbscript"" src="", resourceURL, @"" ></script>");
}
return String.Empty;
}
/// <summary>
/// Let's keep Whidbey name
/// </summary>
/// <param name="originalName"></param>
/// <returns></returns>
public static string GetWebResourceURL(string originalName, ContentType contentType)
{
int index = originalName.LastIndexOf(".");
if (index == -1)
return originalName;
// Read version from <appSettings> section of Web.Config
string version = System.Configuration.ConfigurationSettings.AppSettings["ResourceVersion"];
if (version.Length == 0)
return originalName;
string resourceURL = String.Format("{0}_{1}_{2}_InfraScriptCache.js",
originalName.Substring(0, index), version, contentType);
return resourceURL;
}
}
public enum ContentType
{
js,
css,
vbs
}
public class WebResourceCollection : CollectionBase
{
public int Add(WebResourceDescription res)
{
return base.List.Add(res);
}
public WebResourceDescription this[int index]
{
get
{
return (WebResourceDescription)base.List[index];
}
set
{
base.List[index] = value;
}
}
}
public class WebResourceDescription
{
private string m_Name;
private ContentType m_ContentType;
public string Name
{
get
{
return this.m_Name;
}
set
{
this.m_Name = value;
}
}
public ContentType ContentType
{
get
{
return this.m_ContentType;
}
set
{
this.m_ContentType = value;
}
}
}
}
Appendix B.
Http Handler
using System;
using System.IO;
using System.Web;
using System.Xml;
using System.Reflection;
using System.Threading;
using System.Globalization;
using System.Security.Principal;
using System.Web.Services.Protocols;
using System.Web.Caching;
using System.Text.RegularExpressions;
namespace HTTPServices
{
/// <summary>
///
/// </summary>
public class ResourceService : IHttpHandler
{
#region Constants
private const string CONTENT_TYPE_GROUP_NAME = "souretype";
private const string HEADER_CACHE_CONTROL = "Cache-Control";
private const string HEADER_EXPIRES = "Expires";
private const string HEADER_LAST_MODIFIED = "Last-Modified";
private const string HEADER_ACCEPT_RANGES = "Accept-Ranges";
private const string HEADER_INFRASCRIPTCACHE = "InfraScriptCache";
private const string HEADER_IF_MODIFIED_SINCE = "If-Modified-Since";
private const string HEADER_CACHE_CONTROL_VALUE = "max-age=31536000";
private const string HEADER_LAST_MODIFIED_VALUE = "Fri, 09 Jun 2000 20:25:23 GMT";
private const string HEADER_ACCEPT_RANGES_VALUE = "bytes";
private const string HEADER_INFRASCRIPTCACHE_VALUE = "True";
private const int HEADER_NOT_MODIFIED_VALUE = 304;
private const string MIME_TYPE_JS = "application/x-javascript";
private const string MIME_TYPE_VBS = "application/octet-stream";
private const string MIME_TYPE_CSS = "text/css";
private const string MIME_TYPE_HTC = "text/x-component";
#endregion
/// <summary>
///
/// </summary>
/// <param name="context"></param>
public void ProcessRequest(HttpContext context)
{
// Indicate that custom resource handler was invoked
// Just for debug or future use
context.Response.AppendHeader(HEADER_INFRASCRIPTCACHE, HEADER_INFRASCRIPTCACHE_VALUE);
// Browser check for newer version of resource
if (context.Request.Headers[HEADER_IF_MODIFIED_SINCE] != null)
{
context.Response.StatusCode = HEADER_NOT_MODIFIED_VALUE;
return;
}
// Map Resource URL to real path ,
// URL form: <file_name>_<version>_<content_type>_InfraScriptCache.js
// RegEx: "_\d*_(?<contenttype>\w*)_InfraScriptCache.js$"
// Original Path: commonfuncs_123_InfraScriptCache.js
// Transformed path: commonfunc.js
Regex regex = new Regex(
String.Concat(@"_\d*_(?<", CONTENT_TYPE_GROUP_NAME,
@">\w*)_InfraScriptCache.js$"),
RegexOptions.IgnoreCase
| RegexOptions.IgnorePatternWhitespace
| RegexOptions.Compiled
);
string originalPath = context.Request.Path;
Match mt = regex.Match(originalPath);
string srcType = mt.Groups[CONTENT_TYPE_GROUP_NAME].Value;
// Get real file name
string path = context.Server.MapPath(
regex.Replace(originalPath, String.Concat(".", srcType)));
// Serve resource
ProcessResourceStream(path, srcType, context);
}
/// <summary>
///
/// </summary>
public bool IsReusable
{
get
{
return true ;
}
}
/// <summary>
///
/// </summary>
/// <param name="path"></param>
/// <param name="srcType"></param>
/// <param name="context"></param>
public static void ProcessResourceStream(string path, string srcType, HttpContext context)
{
// Set content type
context.Response.ContentType = GetContentType(srcType);
// Look for resource in cache
byte[] outputBinaryArray = (byte[])context.Cache.Get(path);
if (outputBinaryArray == null)
{
// Create the reader for data.
FileStream fs = new FileStream(path, FileMode.Open, FileAccess.Read);
BinaryReader r = new BinaryReader(fs);
// Read data from file.
outputBinaryArray = r.ReadBytes(Convert.ToInt32(fs.Length));
//cleanup
r.Close();
fs.Close();
//insert into cache
context.Cache.Insert(path, outputBinaryArray);
}
//append needed headers
context.Response.AppendHeader(HEADER_CACHE_CONTROL,HEADER_CACHE_CONTROL_VALUE);
context.Response.AppendHeader(HEADER_EXPIRES,DateTime.Now.AddYears(1).ToUniversalTime().ToLongDateString());
context.Response.AppendHeader(HEADER_LAST_MODIFIED, HEADER_LAST_MODIFIED_VALUE);
context.Response.AppendHeader(HEADER_ACCEPT_RANGES,HEADER_ACCEPT_RANGES_VALUE);
// write output
context.Response.BinaryWrite(outputBinaryArray);
}
/// <summary>
/// Translate code from URL into MIME content type
/// </summary>
/// <param name="srcType"></param>
/// <returns></returns>
private static string GetContentType(string srcType)
{
string mimeType = String.Empty;
// Get Content Type
switch (srcType)
{
case "js" :
mimeType = MIME_TYPE_JS;
break;
case "css" :
mimeType = MIME_TYPE_CSS;
break;
case "htc" :
mimeType = MIME_TYPE_HTC;
break;
case "vbs" :
mimeType = MIME_TYPE_VBS;
break;
default:
mimeType = string.Empty;
break;
}
return mimeType;
}
}
}
Appendix C.Do not forget Web.Config settings:
<httpHandlers>
<add verb="*" path="*_InfraScriptCache.js" type="HTTPServices.ResourceService"/>
</httpHandlers>