[Chugalug] spiderring a site question
Robert A. Kelly III
bluethegrappler at gmail.com
Sat Nov 9 22:29:08 UTC 2013
On 11/03/2013 12:23 PM, Rod wrote:
> I'm thinking of mirroring a site that is about to shutdown. But first I
> want to determine what kind of storage I need.
> My normal tool is wget. But I don't see a way to gauge file/site size
> without downloading. Any ideas on how spider a site and determine file
> size without downloading?
wget has an option specifically for this. It's actually called "--spider".
More information about the Chugalug