Re: OT: wget bug
- From: "Joe R. Jah" <jjah@xxxxxxxxxxxxxxxxxxx>
- Date: Sat, 18 Jul 2009 23:11:58 -0700 (PDT)
On Sat, 18 Jul 2009, Andrew Brampton wrote:
Date: Sat, 18 Jul 2009 18:09:54 +0100
From: Andrew Brampton <brampton+freebsd@xxxxxxxxx>
To: Joe R. Jah <jjah@xxxxxxxxxxxxxxxxxxx>
Subject: Re: OT: wget bug
2009/7/18 Joe R. Jah <jjah@xxxxxxxxxxxxxxxxxxx>:
Thank you Andrew. Yes the server is truly returning 401. I have already
reconfigured wget to download everything regardless of their timestamp,
but it's a waste of bandwidth, because most of the site is unchanged.
Do you know of any workaround in wget, or an alternative tool to ONLY
download newer files by http?
There are two ways to check if the file has been changed. One, read
the time the file was last changed, or two, read the file and compare
it to a old copy. Wget was obviously trying to do option 1 but this is
denied by the remote server. You most likely could get it to do option
2, however by doing so you are wasting bandwidth downloading unchanged
files just to check if they had been changed.
If you have control over the remote webserver, then the simplest way
to solve this problem is to configure the webserver not to return 401
when wget sends the If-Modified-Since HTTP header. A better solution,
again assuming you have control of the remote server, is to use
"rsync" as it is designed for this kind of task.
If you don't have control over the remote server, then you are stuck
with your current solution.
Thank you Andrew.
_/ _/_/_/ _/ ____________ __o
_/ _/ _/ _/ ______________ _-\<,_
_/ _/ _/_/_/ _/ _/ ......(_)/ (_)
_/_/ oe _/ _/. _/_/ ah jjah@xxxxxxxxxxxxxxxxxxx_______________________________________________
freebsd-questions@xxxxxxxxxxx mailing list
To unsubscribe, send any mail to "freebsd-questions-unsubscribe@xxxxxxxxxxx"
- Prev by Date: OT: where to buy Atheros mini pci-exress?
- Next by Date: Re: OT: wget bug
- Previous by thread: Re: OT: wget bug
- Next by thread: Re: OT: wget bug