Wget - Revision history

Leo: Leo moved page Wget Tips & Tricks to Wget: General information on wget

2021-12-30T01:36:40Z

Leo moved page Wget Tips & Tricks to Wget: General information on wget

← Older revision		Revision as of 01:36, 30 December 2021
(No difference)

2020-01-11T19:09:40Z

← Older revision		Revision as of 19:09, 11 January 2020
Line 12:		Line 12:

	Note: This is equivalent to using curl with no options. Eg: {{code\|<nowiki>curl http://google.ca</nowiki>}}.		Note: This is equivalent to using curl with no options. Eg: {{code\|<nowiki>curl http://google.ca</nowiki>}}.

			== Spoofing Gootlebot ==
			Some sites contain a paywall but can be viewed if spoofing Googlebot. To spoof Googlebot, pass in a Googlebot user agent and optionally a {{code\|X-Forwarded-For}} header from a Googlebot IP address.

			{{highlight\|lang=terminal\|code=
			$ wget --header="X-Forwarded-For: 66.249.66.1" --user-agent="Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
			}}

	== Save to Directory ==		== Save to Directory ==

2019-09-01T06:20:40Z

Text replacement - "Category:Linux{{Navbox Linux}}" to "{{Navbox Linux}}Category:Linux"

2019-09-01T05:10:26Z

Text replacement - "{{Navbox Linux}}{{Navbox Linux}}" to "{{Navbox Linux}}"

← Older revision		Revision as of 05:10, 1 September 2019
Line 63:		Line 63:
	* [[aria2c]]		* [[aria2c]]

	[[Category:Linux]]~~{{Navbox Linux}}~~{{Navbox Linux}}		[[Category:Linux]]{{Navbox Linux}}
	[[Category:FreeBSD]]		[[Category:FreeBSD]]
	[[Category:TipsnTricks]]		[[Category:TipsnTricks]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]

2019-09-01T05:08:58Z

Text replacement - "Category:Linux" to "Category:Linux{{Navbox Linux}}"

← Older revision		Revision as of 05:08, 1 September 2019
Line 63:		Line 63:
	* [[aria2c]]		* [[aria2c]]

	[[Category:Linux]]{{Navbox Linux}}		[[Category:Linux]]{{Navbox Linux}}{{Navbox Linux}}
	[[Category:FreeBSD]]		[[Category:FreeBSD]]
	[[Category:TipsnTricks]]		[[Category:TipsnTricks]]
	[[Category:LinuxUtilities]]		[[Category:LinuxUtilities]]

2019-09-01T04:55:46Z

Text replacement - " {{Navbox Linux}}" to ""

2019-09-01T03:48:52Z

Text replacement - "Category:Linux " to "Category:Linux{{Navbox Linux}} "

2019-09-01T01:33:15Z

Text replacement - "{\| " to "{| class="wikitable" "

2019-08-13T22:59:43Z

Saving an Entire Open Directory

← Older revision		Revision as of 22:59, 13 August 2019
Line 40:		Line 40:
	\| No host directory		\| No host directory
	\|-		\|-
	\| {{code\|--cut-dirs=1}}		\| {{code\|1=--cut-dirs=1}}
	\| Remove the 'remote/' directory from the download destination, 1 level.		\| Remove the 'remote/' directory from the download destination, 1 level.
	\|-		\|-

2019-08-13T22:59:33Z

Saving an Entire Open Directory

← Older revision		Revision as of 22:59, 13 August 2019
Line 23:		Line 23:
	To download an entire open directory from a remote server:		To download an entire open directory from a remote server:
	{{highlight\|lang=terminal\|code=		{{highlight\|lang=terminal\|code=
	$ wget --no-check-certificate -np -r -c --reject "index.html*" -e robots=off http://server/remote/dir		$ wget --no-check-certificate -np -nH -r -c --reject "index.html*" -e robots=off http://server/remote/dir
	}}		}}

Line 36:		Line 36:
	\| {{code\|-np}}		\| {{code\|-np}}
	\| No parent directory (don't crawl up)		\| No parent directory (don't crawl up)
			\|-
			\| {{code\|-nH}}
			\| No host directory
			\|-
			\| {{code\|--cut-dirs=1}}
			\| Remove the 'remote/' directory from the download destination, 1 level.
	\|-		\|-
	\| {{code\|-r}}		\| {{code\|-r}}