Google Suggest seems to be the topic du jour in the blogosphere. It is a cool feature, but what I really enjoy is that it is yet another real world example of a “chubby” client.
I gave a presentation at an NNUG meeting at Microsoft’s Oslo office two months ago on how to create features akin to the auto-complete feature on display over at Google. In this article I will answer the question “everybody” is asking – How do did the guys at Google do that?
Out-of-band calls
It all started when Netscape released Navigator 2.0 in 1996. This browser boosted two revolutionary new features; frames and JavaScript. Although Netscape had different intensions for these features, developers could exploit frames and JavaScript to set up an out-of-band communications channel between the browser and the web server. This made it possible to update forms data without posting back. The concept behind the communications channel was simple. You created a frameset made up of two frames. One of the frames had the height or width attribute set to zero, in practice making the frame “invisible”. The other frame contained the user interface. When the user clicked a button she triggered a JavaScript function which replaced the location property of the hidden frame with the URL for a CGI script that handled the request. The various parameters were passed thru the query string.
top.RPCFrm.location.replace("RPC.cgi?CustomerId="+CustomerId.value);
When the page in the “hidden” frame was refreshed, the BODY element’s onload event was used to pass the result from the CGI script to a callback function.
function callback(result) {
fields=result.split(',');
Firstname.value=fields[0];
MiddleName.value=fields[1];
Surname.value=fields[2];
Address.value=fields[3];
PostalCode.value=fields[4];
City.value=fields[5];
Country.value=fields[6];
}
At the end of the 1990ies Microsoft introduced a technology called remote scripting. This was a combination of JavaScript helper methods, a proxy implemented as a Java applet and some ASP code which enabled you to do both synchronous and asynchronous calls from client side script to functions in server side ASP pages. Now developers could call server side methods via the Java proxy using the RSExecute method.
function LookUpBtn_onclick() {
var callObject = RSExecute("RemoteScriptingFacade.aspx","GetCustomer",CustomerId.value);
fields=callObject.return_value.split(',');
Firstname.value=fields[0];
MiddleName.value=fields[1];
Surname.value=fields[2];
Address.value=fields[3];
PostalCode.value=fields[4];
City.value=fields[5];
Country.value=fields[6];
}
The above example shows a synchronous remote scripting method invocation. Remote scripting relied on Java, which is the key technology of Microsoft’s nemesis. On top of that even if the protocol was XML like, it was a proprietary protocol. Therefore when SOAP was introduced in 1998, Microsoft released the Web Service Behavior for Internet Explorer. The behavior leveraged the standardized SOAP protocol and Internet Explorer’s built-in XmlHttp support. As the name implies, the component enabled developers to invoke Web Service methods from JavaScript code. The example below shows how a Web Service can be invoked asynchronously from the browser.
function LookUpBtn_onclick() {
WebService.facade.callService(callback,"GetCustomer",CustomerId.value);
}
function callback(result) {
if (result.error) alert("WS Error:\nCode: "+result.errorDetail.code+"\nString:\n"+result.errorDetail.string+"\nRaw:\n"+result.errorDetail.raw);
else {
Firstname.value=result.raw.selectSingleNode("//FirstName").text;
MiddleName.value=result.raw.selectSingleNode("//MiddleName").text;
Surname.value=result.raw.selectSingleNode("//Surname").text;
Address.value=result.raw.selectSingleNode("//Address").text;
PostalCode.value=result.raw.selectSingleNode("//PostalCode").text;
City.value=result.raw.selectSingleNode("//City").text;
Country.value=result.raw.selectSingleNode("//Country").text;
}
}
With the release of the .NET framework, attention shifted from browser based applications towards richer Windows Forms clients. Today, sophisticated web applications such as GMail, A9 and most recently Google Suggest has drawn attention to back to “chubby” clients combining some of the features in “fat” clients with the benefits of browser based applications. Microsoft is not by any means behind on this development. ASP.NET 2.0 has a new feature called script callbacks which relies on XmlHttpRequest to call back to the server.
public partial class Default_aspx : ICallbackEventHandler {
private string callbackScriptBlock=String.Empty;
public string CallbackScriptBlock
{
get
{
return callbackScriptBlock;
}
}
public void Page_Load(object sender, System.EventArgs e)
{
callbackScriptBlock = this.GetCallbackEventReference(this, "customerId", "showCustomerDetails", "context", "onError");
}
public string RaiseCallbackEvent(string eventArg)
{
Customer.Path = Server.MapPath("../Data/Customers.xml");
Customer customer = Customer.Load(eventArg);
return Customer.ToCsvString(customer);
}
}
The above example shows how to create a page that implements the ICallBackEventHandler interface. Below are the JavaScript methods from the ASP.NET page.
function LookUpBtn_onclick() {
var context="";
var customerId=form1.CustomerId.value;
<%= this.CallbackScriptBlock %>
}
function showCustomerDetails(result, context) {
fields=result.split(',');
form1.Firstname.value=fields[0];
form1.MiddleName.value=fields[1];
form1.Surname.value=fields[2];
form1.Address.value=fields[3];
form1.PostalCode.value=fields[4];
form1.City.value=fields[5];
form1.Country.value=fields[6];
}
The Google solution
All of the techniques above enable out-of-band communications between the browser and the server. The million dollar question is what is under Google Suggest’s hood?
First Google Suggest will try to instantiate Microsoft’s XmlHttp component using various references. If this doesn’t succeed an attempt is made to instantiate XMLHttpRequest. Microsoft first implemented the XMLHttpRequest object in Internet Explorer 5 for Windows as an ActiveX object. Engineers on the Mozilla project implemented a compatible native version for Mozilla 1.0. Apple has done the same starting with Safari 1.2. The XmlHttp or XMLHttpRequest object is used to make HTTP get requests to http://www.google.com/complete/search passing the text in the search field, the language and a JavaScript flag as query string parameters. Below is the response when searching for “XmlHttpRequest”.
sendRPCDone(frameElement, "xmlhttprequest", new Array("xmlhttprequest", "xmlhttprequest post", "xmlhttprequest javascript", "xmlhttprequest mozilla", "xmlhttprequest example", "xmlhttprequest send", "xmlhttprequest object", "xmlhttprequest php", "xmlhttprequest timeout", "xmlhttprequest firefox"), new Array("52,800 results", "6,010 results", "28,400 results", "15,900 results", "3,780 results", "8,860 results", "7,380 results", "46,100 results", "220 results", "10,500 results"), new Array(""));
As you can see, JavaScript code is returned. This code is executed using the eval method. Click here to submit a query for XmlHttpRequest to Google Suggest.
If Google Suggest was unable to create an XmlHttp or XMLHttpRequest instance it falls back to an “advanced” use of the oldest trick in the book; it dynamically creates an IFRAME which is used to phone home. The URL is the same as when using XmlHttp but the JavaScript flag set to false. Below is the response from a fallback request.
<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<script>
function bodyLoad() {
if (parent == window) return;
var frameElement = this.frameElement;
parent.sendRPCDone(frameElement, "xmlhttprequest", new Array("xmlhttprequest", "xmlhttprequest post", "xmlhttprequest javascript", "xmlhttprequest mozilla", "xmlhttprequest example", "xmlhttprequest send", "xmlhttprequest object", "xmlhttprequest php", "xmlhttprequest timeout", "xmlhttprequest firefox"), new Array("52,800 results", "6,010 results", "28,400 results", "15,900 results", "3,780 results", "8,860 results", "7,380 results", "46,100 results", "220 results", "10,500 results"), new Array(""));
}
</script></head><body onload='bodyLoad();'></body></html>
To achieve the super fast responses, Google does not perform an actual search every time a character is entered in the query field. Instead Google keeps an in-memory list of the most popular searches and returns suggestions from this list.
ASP.NET Suggest
The Google Suggest feature is fairly simple to implement using ASP.NET 2.0 script callbacks. The example is close to the one above with some minor differences. Instead of making an out-of-band call when the user clicks a button, a call is made every time the user enters a character in the search field. To do this you simply hook your remote method invocation stub to the onkeyup event on the INPUT element.
<input onkeyup="GetAutoComplete();" autocomplete="off" id="Query" type="text" size="40" />
You should also switch off the browser’s auto-complete feature; after all you don’t want it to compete with your far superior suggestion feature.
The RaiseCallbackEvent method in the code behind class should simple retrieve all popular searches starting with the same character sequences as the user’s current query.
public string RaiseCallbackEvent(string query)
{
List<string> results = popularSearches.FindAll(delegate(string s)
{
return s.StartsWith(query);
});
StringBuilder sb = new StringBuilder();
foreach (string result in results)
{
sb.Append(result);
sb.Append(',');
}
sb.Append(' ');
return sb.ToString();
}
The JavaScript callback handler then needs to display the relevant suggestions to the user.
function fillAutoComplete(result, context) {
fields=result.split(',');
form1.AutoComplete.value='';
for (var i=0; i<fields.length; i++) {
form1.AutoComplete.value+=fields[i]+'\n';
}
}
I’ve chosen to do a rather naïve implementation of the suggestion feature to keep my sample easy to follow. To achieve a googlesque look, with all its bells and whistles, simply extend the above method to use some DHTML magic.
The complete source code can be viewed here: Default.aspx and Default.aspx.cs