Wget: Difference between revisions

From Leo's Notes
This page was last edited on 6 March 2017, at 00:31.
Line 23: Line 23:
To download an entire open directory from a remote server:
To download an entire open directory from a remote server:
{{highlight|lang=terminal|code=
{{highlight|lang=terminal|code=
wget -np -r http://server/remote/dir
$ wget --no-check-certificate -np -r -c --reject "index.html*" -e robots=off http://server/remote/dir
}}
}}


Where:
{|
! Flag
! Descrption
|-
| {{code|--no-check-certificate}}
| Don't check SSL certificates
|-
| {{code|-np}}
| No parent directory (don't crawl up)
|-
| {{code|-r}}
| Recursive
|-
| {{code|-c}}
| Continue
|-
| {{code|--reject "index.html*"}}
| Don't save index.html* files from directory listing
|-
| {{code|-e robots=off}}
| Don't check robots.txt
|}


== wget on FreeBSD ==
== wget on FreeBSD ==

Revision as of 00:31, 6 March 2017

wget to console

Use the -O - option to output contents to the console.

wget -O - http://google.ca

Since the contents go to stdout, you can pipe it to a shell to execute remote commands:

wget -O - http://scripthost.local/fix_graphics.sh | sh

Note: This is equivalent to using curl with no options. Eg: curl http://google.ca.

Save to Directory

Use the -P to specify the prefix of the output file.

wget -P /tmp http://google.ca/


Saving an Entire Open Directory

To download an entire open directory from a remote server:

$ wget --no-check-certificate -np -r -c --reject "index.html*" -e robots=off http://server/remote/dir

Where:

Flag Descrption
--no-check-certificate Don't check SSL certificates
-np No parent directory (don't crawl up)
-r Recursive
-c Continue
--reject "index.html*" Don't save index.html* files from directory listing
{{{1}}} Don't check robots.txt

wget on FreeBSD

wget is available in ports under ftp/wget. If you don't want to compile it, you could just use fetch (1) to download files.

See Also