[Chugalug] spiderring a site question

Dave Brockman dave at brockmans.com
Sun Nov 3 17:42:10 UTC 2013


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 11/3/2013 12:23 PM, Rod wrote:
> I'm thinking of mirroring a site that is about to shutdown. But
> first I want to determine what kind of storage I need. My normal
> tool is wget. But I don't see a way to gauge file/site size without
> downloading. Any ideas on how spider a site and determine file size
> without downloading?
> 

https://github.com/mariomaric/website-size

?

Regards,

dtb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.17 (MingW32)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJSdorxAAoJEMP+wtEOVbcd7N8IAKpuCD9X3me+o9bA6QKx0Qmd
KjaBim7DFLRM7ced72iti3OpEh41d9Nzy0ibl8tE8w5vA1wippJU6olA4qGlfH6v
51oP/YYkV/vcmqwJzz2wZxeVKaQwKI7kFCX8wLCVFToi+tuGkM/Ro0iNJq+pA5IV
4VeeJJ/8YMLkmD1UHPehXICNTS0DMb8EwChD83pi/e6h7h5l+G3cAyNcD3Mzrr9S
2HzKxOefkgFL4QMw5RhrQDxTqaOtnzFTBm0FJpxrp5vQxdcvEctCOyM2p9mRHmM3
Tayvhd9yAN/eRLTROQlGjDiMB1fwZMJ7coBDa8semHq5t5A7Z01QqIHYZv4dMFw=
=R5j2
-----END PGP SIGNATURE-----


More information about the Chugalug mailing list