Sunday, September 30, 2007

Download an XML feed with wget

For one of my projects, I was attempting to download a 70MB XML feed. Unfortunately, it was taking forever to download via HTTP and crashing my MSIE 6.0 and Firefox browsers. I ended up using the wget utility to complete the task. Here's what I did.

C:\Documents and Settings\esoft\Desktop>wget
=> `esofthub xmllists.xml'
Resolving xx.1xx.x.2xx
Connecting to|xx.1xx.x.2xx|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 70,722,387 (67M) [text/xml]

100%[====================================>] 70,722,387 130.78K/s ETA 00:00

02:08:32 (182.75 KB/s) - `esofthub xmllists.xml' saved [70722387/70722387]

If you don't have wget, here's a couple site you can go to get it.

Download WGET for Windows
Download GNU WGET


mike said...

Sometimes axel is faster. '[I]t opens more than one HTTP/FTP connection per download and each connection transfers its own, separate, part of the file.'

esofthub said...


Interesting. I'll have to try that. Thanks.