wget
-
- Contributor
- Posts: 1857
- Joined: 09 Aug 2013, 14:25
- Distribution: Porteus and Nemesis
- Location: USA
wget
Anyone have a script that use wget to download a lot of files at one time from the same place?
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.
-
- Full of knowledge
- Posts: 2071
- Joined: 17 Jun 2013, 13:17
- Distribution: Porteus 3.2.2 XFCE 32bit
- Location: Germany
wget
Hi Jack
you could try
In this example, -r and -l1 options enable 1-level deep recursive retrieval,
and "-A" option specifies file name suffix to accept during recursive download.
(tgz in this case)
EDIT
If needed add the -np, --no-parent switch = don't ascend to the parent directory
you could try
Code: Select all
wget -r -l1 -A.tgz http://whatever.com/directory/
and "-A" option specifies file name suffix to accept during recursive download.
(tgz in this case)
EDIT
If needed add the -np, --no-parent switch = don't ascend to the parent directory
-
- Shogun
- Posts: 206
- Joined: 18 Aug 2013, 12:09
- Distribution: Slackware PartedMagic Xubuntu
- Location: The Netherlands
wget
In a 1-line script I have:
where the "$@" represents the "http://<dir>" or "https://<dir>" invocation parameter(s).
Code: Select all
wget -m -e robots=off --no-parent --reject="index.html*" "$@"
- brokenman
- Site Admin
- Posts: 6105
- Joined: 27 Dec 2010, 03:50
- Distribution: Porteus v4 all desktops
- Location: Brazil
wget
Just for variation.
Burdi01's line above should also do it nicely.
Code: Select all
for a in $(lynx -dump --listonly --nonumbers http://brokenman.porteus.org/current/x86_64/custom | egrep "txz|tgz|xzm"); do
wget $a
done
How do i become super user?
Wear your underpants on the outside and put on a cape.
Wear your underpants on the outside and put on a cape.
- Ed_P
- Contributor
- Posts: 8369
- Joined: 06 Feb 2013, 22:12
- Distribution: Cinnamon 5.01 ISO
- Location: Western NY, USA
wget
Why download all of them rather than just the ones you want/need??
-update-
And now I know.
-update-
And now I know.
Jack wrote: ↑06 Nov 2017, 19:28here is the link. http://slackware.uk/msb/current/1.19/x86_64/ The directory are base, deps and extra just the txz files.
Ed
-
- Full of knowledge
- Posts: 2071
- Joined: 17 Jun 2013, 13:17
- Distribution: Porteus 3.2.2 XFCE 32bit
- Location: Germany
wget
Code: Select all
wget -r -l1 -A.tgz,txz,xzm http://brokenman.porteus.org/current/x86_64/custom/
-
- Contributor
- Posts: 1857
- Joined: 09 Aug 2013, 14:25
- Distribution: Porteus and Nemesis
- Location: USA
wget
Donald, I did what you first posted and it work great. And I did it before burdi01 and brokenman posted but I will try them. Now I see you posted another way to do it.
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.
-
- Contributor
- Posts: 1857
- Joined: 09 Aug 2013, 14:25
- Distribution: Porteus and Nemesis
- Location: USA
wget
Well I did a test on all the ways posted and they all work well so I will have to pick one.
I like to be able to download what in these directory here is the link. http://slackware.uk/msb/current/1.19/x86_64/ The directory are base, deps and extra just the txz files.
I like to be able to download what in these directory here is the link. http://slackware.uk/msb/current/1.19/x86_64/ The directory are base, deps and extra just the txz files.
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.
-
- Contributor
- Posts: 1857
- Joined: 09 Aug 2013, 14:25
- Distribution: Porteus and Nemesis
- Location: USA
wget
I want to do these in one line if it can be done. Or a script will do to. I don't like to download one file at a time.Ed_P wrote: ↑06 Nov 2017, 15:53Why download all of them rather than just the ones you want/need??
-update-
And now I know.Jack wrote: ↑06 Nov 2017, 19:28here is the link. http://slackware.uk/msb/current/1.19/x86_64/ The directory are base, deps and extra just the txz files.
Mate Files
wget -r -l1 -A.txz http://slackware.uk/msb/current/1.19/x86_64/base
wget -r -l1 -A.txz http://slackware.uk/msb/current/1.19/x86_64/deps
wget -r -l1 -A.txz http://slackware.uk/msb/current/1.19/x86_64/extra
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.
-
- Full of knowledge
- Posts: 2071
- Joined: 17 Jun 2013, 13:17
- Distribution: Porteus 3.2.2 XFCE 32bit
- Location: Germany
wget
I do use a "list.txt" file then.
A simple txt file containing the URL's
Code: Select all
http://slackware.uk/msb/current/1.19/x86_64/base
http://slackware.uk/msb/current/1.19/x86_64/deps
http://slackware.uk/msb/current/1.19/x86_64/extra
Code: Select all
guest@porteus:~/Downloads$ wget -r -l1 -A.txz -i ./list.txt
-
- Contributor
- Posts: 1857
- Joined: 09 Aug 2013, 14:25
- Distribution: Porteus and Nemesis
- Location: USA
wget
Here is the output something must be missing? Where do I get list.txt at?
Code: Select all
guest@porteus:~/Downloads$ wget -r -l1 -A.txz -i ./list.txt
./list.txt: No such file or directory
No URLs found in ./list.txt.
guest@porteus:~/Downloads$
I just like Slackware because I think it teach you about Linux to build packages where Ubuntu is like Windows you just install programs you want.