Wget: Difference between revisions
From Leo's Notes
This page was last edited on 6 March 2017, at 00:31.
| Line 23: | Line 23: | ||
To download an entire open directory from a remote server: | To download an entire open directory from a remote server: | ||
{{highlight|lang=terminal|code= | {{highlight|lang=terminal|code= | ||
$ wget --no-check-certificate -np -r -c --reject "index.html*" -e robots=off http://server/remote/dir | |||
}} | }} | ||
Where: | |||
{| | |||
! Flag | |||
! Descrption | |||
|- | |||
| {{code|--no-check-certificate}} | |||
| Don't check SSL certificates | |||
|- | |||
| {{code|-np}} | |||
| No parent directory (don't crawl up) | |||
|- | |||
| {{code|-r}} | |||
| Recursive | |||
|- | |||
| {{code|-c}} | |||
| Continue | |||
|- | |||
| {{code|--reject "index.html*"}} | |||
| Don't save index.html* files from directory listing | |||
|- | |||
| {{code|-e robots=off}} | |||
| Don't check robots.txt | |||
|} | |||
== wget on FreeBSD == | == wget on FreeBSD == | ||
Revision as of 00:31, 6 March 2017
wget to console
Use the -O - option to output contents to the console.
wget -O - http://google.ca
Since the contents go to stdout, you can pipe it to a shell to execute remote commands:
wget -O - http://scripthost.local/fix_graphics.sh | sh
Note: This is equivalent to using curl with no options. Eg: curl http://google.ca.
Save to Directory
Use the -P to specify the prefix of the output file.
wget -P /tmp http://google.ca/
Saving an Entire Open Directory
To download an entire open directory from a remote server:
$ wget --no-check-certificate -np -r -c --reject "index.html*" -e robots=off http://server/remote/dir
Where:
| Flag | Descrption |
|---|---|
--no-check-certificate
|
Don't check SSL certificates |
-np
|
No parent directory (don't crawl up) |
-r
|
Recursive |
-c
|
Continue |
--reject "index.html*"
|
Don't save index.html* files from directory listing |
{{{1}}}
|
Don't check robots.txt |
wget on FreeBSD
wget is available in ports under ftp/wget. If you don't want to compile it, you could just use fetch (1) to download files.