Gavin Donoho

Unzipping in memory

Recently I had an assignment to unzip an xml file in memory and save the contents to a local database. The requirement was to pull a keyword performance report from MSN adCenter and save the details to a local database. These reports are generated on the MSN server by making a web service call. After the report is generated, the web service returns the url of a zipped copy of the report. The existing solution would save this zip file to the file system, unzip it, and save the contents to the local database. However, I knew it could be done using a memory stream.

Here is the code to retrive the report from MSN. "reportURL" is a string that is returned from the MSN web service.

HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(reportURL);
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();

Using SharpZipLib, I then handled the unzipping of the file into memory. I create an ZipInputStream object and pass in the Response Stream generated in the above code.

ZipInputStream zipStream = new ZipInputStream(webResponse.GetResponseStream());
ZipEntry zipFile = zipStream.GetNextEntry();

byte[] buffer = new byte[2048];
MemoryStream memoryStream = new MemoryStream();
int read;

while (true)
{    read = zipStream.Read(buffer, 0, buffer.Length);
   if (read <= 0) break;
   memoryStream.Write(buffer, 0, read);
}
memoryStream.Position = 0;

Now I have the unzipped file stored in a memory stream. In the final step, I simply create a new dataset and read the xml from the memory stream. This allows me to loop through the records and insert records into my local database. Not all of the content of this report will be stored. An alternative would be to create an in-memory xml document and use XPath to extract the necessary data.

DataSet reportDataSet = new DataSet();
reportDataSet.ReadXml(memoryStream);

No extraneous files left on the hard drive. Works like a charm! I hope this helps someone out because it took a while for me to put the pieces together.