Getting Remote Pages with ASP
(Page 1 of 4 )
This article will explain step-by-step how to retrieve a remote web page using ASP, save it locally, and avert all kinds of disaster along the way. As a bonus, I’ll provide a demonstration of how to parse our saved file for the information we really want.
Perusing the posts on the ASP Free forums, I've stumbled upon a number of similarly interesting requests. My first reaction to these requests was to reminisce, to think about hockey pools and fine dining. My second reaction was to answer the posts. My third, following the second by mere milliseconds, was the realization that the answer will take up a couple of pages, so why not write an article on the subject, so that all can benefit?
To clarify: a number of times now I've seen people asking the same question: how the heck can I use ASP (or even just VBS) to retrieve the contents of a web page and save it locally?
Well it so happens that at one time I was faced with that precise problem. The story goes like this: One day, I'm coding away happily, and one of my coworkers came to me in absolute distress! He was in charge of running the hockey pool at work, and a serious issue had arisen. Previously the website offering the hockey stats had provided a CSV file for download, but upon restructuring the site, had begun to provide the stats via a web page only. It was absolutely critical to have the stats in CSV format, and he promised me and my girlfriend (now wife) dinner at the fine restaurant of our choosing if I could deliver them.
Now your needs may pale in comparison to the importance of a hockey pool (that was sarcasm...), but I promise you that after reading, your inability to step down from a challenge will be rewarded by the tools and the know-how to accomplish said task. Oh, and I'll also show you how I parsed the file after saving it.
I have used this method quite successfully and efficiently in a publishing process built into my Content Management System, but once again, not nearly as important as hockey statistics.
Keep in mind please, that at the time web services were not available for use. Also, this was classic ASP that I was using, not .NET, with which you handle this task much differently.
Before We Proceed
This article will make use of the MSXML2.ServerXMLHTTP object as well as the Proxy Configuration Tool provided by Microsoft. Make sure you have both of these installed on the machine from which the script is running. If through ASP, they need to be installed on the server, and if through VBS, on your client machine.
You can find the Microsoft XML parser 3.0 sp4 here (http://www.microsoft.com/downloads/details.aspx?FamilyId=C0F86022-2D4C-4162-8FB8-66BFC12F32B0&displaylang=en), and the proxy configuration tool here (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/winhttp/http/proxycfg_exe__a_proxy_configuration_tool.asp).
Next: Let's Code >>
More ASP Articles
More By Justin Cook
/ 46 
