Page 3 of 5
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 23 Jun 2017, 23:59
by Bogomips
@ Ed
Seeing as you sort of owe me big time, please do get cracking and testing
badm.sh
Code: Select all
#!/bin/bash
# badm.sh Display n worst mirrors in repo by Bogomips 2017-06-23 14:54:17
# Repo defaults to 'slackware'
# Arbitrary cutoff points: n<100 & (# of mirrors - n) > 9
badm ()
{
um="Argument usage:\t< # of Worst Mirrors to Display >\t[Repository(default:slackware)]"
err="Error! "
ms=("# requested $n > 100" "Requested 0 worst mirrors" "Non-numeric 1st argument: '$1'" "Repo not found: '$rpo'" "Less than 10 remaining mirrors: $mc-$n=$rm" "$rpo.lst not found. Ensure ${rpo^^} has been set thru fastest mirror script ");
local r=0 q=0;
let y=1 n=-1
[[ $1 ]] || { echo -e $um; return 0; }
[[ $1 =~ ^[0-9]+$ ]] && let n=$1;
[[ n -gt 100 ]] && let r=y; let y++;
((n)) || let r=y; let y++;
[[ n -lt 0 ]] && let r=y; let y++; # Non-numeric 1st Argument
# Default slackware?
# 2nd Argument: Repo (default: slackware)
rpo=${2:-slackware};
# let z=y; # Set 2nd Arg error message level
# Check the Repository
eval $(grep DISTROS= /etc/usm/usm.conf); # Get distros list from usm.conf
[[ $DISTROS ]] || { echo Error\! File usm.conf corrupted; return 9; }
DISTROS+=" sbo"; # 2017-06-19 23:44:55
[[ $DISTROS =~ ([[:space:]]|^)$rpo([[:space:]]|$) ]] || let q=y; let y++;
# Check n ag mirror count
# Get mirror count
if ! ((r)); then
mc=$(sed /Slackware64-current/q /etc/usm/mirrors-$rpo.txt | grep -Ec "^\s*#?\s*(http|ftp)");
# Check if remaining mirros exceed 9
let rm=mc-n; [[ rm -lt 10 ]] && let r=y; let y++;
[[ -f /tmp/$rpo.lst ]] || let r=y; let y++;
if ! ((r)); then echo "# The $n worst mirrors of ${rpo^^}"
cut -d" " -f2- /tmp/sesm/$rpo.lst | sort -k2 | cut -d" " -f1 | tail -n$n
return 0;
fi
fi
# Error Message Output
((r)) && echo $err${ms[r]}
((q)) && echo $err${ms[r]}
return ${q:-$r};
}
badm $*
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 24 Jun 2017, 03:04
by Ed_P
Needs work.
Code: Select all
guest@porteus:~$ sh ./USM/badm.sh
Argument usage: < # of Worst Mirrors to Display > [Repository(default:slackware)]
guest@porteus:~$ sh ./USM/badm.sh 5
Error!
guest@porteus:~$ sh ./USM/badm.sh 5 salix
Error!
guest@porteus:~$ sh ./USM/badm.sh 5 slackware
Error!
guest@porteus:~$
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 24 Jun 2017, 21:10
by Bogomips
Thanks, Ed, Did not expect rest of error message to not show.
Error message should now show.
badm.shCode: Select all
#!/bin/bash
# badm.sh Display n worst mirrors in repo by Bogomips 2017-06-23 14:54:17
# Repo defaults to 'slackware'
# Arbitrary cutoff points: n<100 & (# of mirrors - n) > 9
# last Update: 2017-06-24 22:19:36 Error Message repetioins fixed
badm ()
{
um="Argument usage:\t< # of Worst Mirrors to Display >\t[Repository(default:slackware)]"
err="Error! "
local r q=0;
let y=1 n=-1
[[ $1 ]] || { echo -e $um; return 0; }
[[ $1 =~ ^[0-9]+$ ]] && let n=$1;
[[ n -gt 100 ]] && let r=y; let y++;
((n)) || let r=y; let y++;
[[ n -lt 0 ]] && let r=y; let y++; # Non-numeric 1st Argument
# Default slackware?
# 2nd Argument: Repo (default: slackware)
rpo=${2:-slackware};
# let z=y; # Set 2nd Arg error message level
# Check the Repository
eval $(grep DISTROS= /etc/usm/usm.conf); # Get distros list from usm.conf
[[ $DISTROS ]] || { echo Error\! File usm.conf corrupted; return 9; }
DISTROS+=" sbo"; # 2017-06-19 23:44:55
[[ $DISTROS =~ ([[:space:]]|^)$rpo([[:space:]]|$) ]] || let q=y; let y++;
# Check non-numeric 2nd Arg
[[ ! $rpo =~ ^[a-z]+$ && n -lt 0 ]] && q=0;
# Check n ag mirror count
# Get mirror count
if ! ((${r:=q})); then
mc=$(sed /Slackware64-current/q /etc/usm/mirrors-$rpo.txt | grep -Ec "^\s*#?\s*(http|ftp)");
# Check if remaining mirros exceed 9
let rm=mc-n; [[ rm -lt 10 ]] && let r=y; let y++;
[[ -f /tmp/sesm/$rpo.lst ]] || let r=y; let y++;
if ! ((r)); then echo "# The $n worst mirrors of ${rpo^^}"
cut -d" " -f2- /tmp/sesm/$rpo.lst | sort -k2 | cut -d" " -f1 | tail -n$n
return 0;
fi
fi
# Error Message Output
ms=("# requested $n > 100" "Requested 0 worst mirrors" "Non-numeric 1st argument: '$1'" "Repo not found: '$rpo'" "Less than 10 remaining mirrors: $mc-$n=$rm" "$rpo.lst not found. Ensure ${rpo^^} has been set thru fastest mirror script");
((r)) && echo $err${ms[r-1]}
[[ q -gt r ]] && echo $err${ms[q-1]}
return ${q:-$r};
}
badm $*
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 24 Jun 2017, 21:23
by Ed_P
Needs work.
Code: Select all
guest@porteus:~$ ./USM/badm.sh
Argument usage: < # of Worst Mirrors to Display > [Repository(default:slackware)]
guest@porteus:~$ ./USM/badm.sh 5
Error! slackware.lst not found. Ensure SLACKWARE has been set thru fastest mirror script
guest@porteus:~$ ./USM/badm.sh 5 /etc/usm/
Error! Repo not found: '/etc/usm/'
guest@porteus:~$ ./USM/badm.sh 5 /etc/usm/slackware.lst
Error! Repo not found: '/etc/usm/slackware.lst'
guest@porteus:~$ ./USM/badm.sh 5 /etc/usm/mirrors-slackware.txt
Error! Repo not found: '/etc/usm/mirrors-slackware.txt'
guest@porteus:~$ ls /etc/usm/mirrors*
/etc/usm/mirrors-alien.txt /etc/usm/mirrors-slackonly.txt
/etc/usm/mirrors-ponce.txt /etc/usm/mirrors-slackware.txt
/etc/usm/mirrors-salix.txt /etc/usm/mirrors-slackwarepatches.txt
/etc/usm/mirrors-sbo.txt /etc/usm/mirrors-slacky.txt
guest@porteus:~$
It appears you are looking for files in the /tmp folder created by masm.sh and before the distro has been rebooted
and the /tmp/ folder cleared.
I suggest you change the default msg to be
"< # of Worst Mirrors to Display > [/tmp Repository(default:slackware.lst)]".
Ideally IMHO a Mirror Settings script that would present the user with the options to:
- select all distro mirrors or just one
- then option to list the 3-5 fastest mirrors for the selected distro's mirror/s or the 3-5 slowest mirrors
- if a single distro was selected and the fastest mirrors option selected the option to choose one of
the displayed mirrors and update the distro's file with it
would be a very useful tool.
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 24 Jun 2017, 23:29
by Bogomips
^ Thanks for feedback. Did not say too much because, needed to see how error message came across. Prior to running badm.sh, if you set the slackware mirrors by running sesm.sh, all the files will be there. i.e. after running sesm all , one can then pick up a list of bad mirrors which do not bear testing.
Anyway one can start small by putting them on command line after all. Expect these to be fastest mirrors with some sort of issue e.g. slackwarecatalogs, which crashes the usm slackware update, and fastest slacky with an issue as well. So that will be next small step.
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 27 Jun 2017, 00:48
by Bogomips
Script to set the Slackware usm Mirrors enhanced with Mirror Exclusion List on Command Line:
Code: Select all
guest@porteus:~$ bash sesm.sh
Argument usage: all [<Toggle Debug: 0/1>] [URL ..]
URL .. Optional Exclusion Mirror List on Command Line
http://forum.porteus.org/viewtopic.php? ... c7a#p55999
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 27 Jun 2017, 03:50
by Ed_P
Code: Select all
guest@porteus:~$ ./USM/sesm.sh
Argument usage: all [<Toggle Debug: 0/1>] [ URL .. ]
URL .. Optional Exclusion Mirror List on Command Line
guest@porteus:~$ ./USM/sesm.sh all slackware
Error! Mirror Parameter neither http nor ftp:
slackware
guest@porteus:~$
It works, I guess.
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 28 Jun 2017, 00:42
by Bogomips
Ed_P wrote:Ideally IMHO a Mirror Settings script that would present the user with the options to:
- select all distro mirrors or just one
- then option to list the 3-5 fastest mirrors for the selected distro's mirror/s or the 3-5 slowest mirrors
-
if a single distro was selected and the fastest mirrors option selected the option to choose one of
the displayed mirrors and update the distro's file with it
would be a very useful tool.
Preoccupied with immediate concerns, but hey, DIY! (It's an IKEA sort of thing):
Mirror Settings script that would present the user with the options to:
- select all distro mirrors or just one
sesm.sh or Set fastest mirror for a repo:
fasm.sh
then option to list the 3-5 fastest mirrors for the selected distro's mirror/s or the 3-5 slowest mirrors
/tmp repo.lst sort and cut or cut and sort, and then head or tail. Worked out code for this in the first post.
if a single distro was selected and the fastest mirrors option selected the option to choose one of
the displayed mirrors and update the distro's file with it
Run fasm.sh then then URLs from /tmp/fasm|main/repo.lst feed into a
select, and after choice made, use manual setting of slackware mirror
masm.sh to set it.
QED
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 01 Jul 2017, 00:34
by Bogomips
Pleased to provide Icing on Cake for Delectation of All and Sundry. 8)
http://forum.porteus.org/viewtopic.php? ... 9fb#p55999
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 01 Jul 2017, 04:17
by Ed_P
Bogomips wrote:Updates
Sat Jul 1
Sat Jul 1 = 2017-07-01. Where is the 2017-07-01 change?
sesm.sh shows Last Update 2017-06-30.
[*]Script
sesm.shCode: Select all
#!/bin/bash
# sesm.sh Set the Slackware Miirors for usm Repos Bogomips 2017-06-13 19:33:50
# fasm.sh Adaptation by Bogomips to Set Fastest Slackware Repository Mirror 2017-06-07 21:35:10
# Last Update 2017-06-30 21:49:19 Comments; bad(): Output msgs + formatting
Bogomips wrote:Cut Out can be made from
sesm.sh for the Script
fasm.sh that sets the Fastest Slackware Mirror for a Repo:
Code: Select all
sed "s/\s*sesm\s*().*/fasm \$*/;T;q" $g/sesm.sh > fasm.sh
guest@porteus:~$ bash fasm.sh
Argument usage: <Repository> [<Toggle Debug: 0/1>] [ URL .. ]
URL .. Optional Exclusion Mirror List on Command Line
Optional Exclusion File of URLs: .usm/badmirrors.txt (following format of 'mirrors-<repo>,txt')
Code: Select all
guest@porteus:~$ bash fasm.sh slacky
All Exclusion URLs:
Cmd Line URLs=0 File (.usm/badmirrors.txt) URLs=64 Merged(Unique) URLs=51
Done!
slacky mirror set to: http://slack.isper.sk/pub
Where is the .usm/ folder for the badmirrors.txt file referenced above suppose to be?
Code: Select all
guest@porteus:~$ ls .usm/
/bin/ls: cannot access '.usm/': No such file or directory
guest@porteus:~$ su
Password:
root@porteus:/home/guest# ls ./usm
/bin/ls: cannot access './usm': No such file or directory
root@porteus:/home/guest#
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 01 Jul 2017, 23:29
by Bogomips
Ed_P wrote:Bogomips wrote:Updates
Sat Jul 1
Sat Jul 1 = 2017-07-01. Where is the 2017-07-01 change?
sesm.sh shows Last Update 2017-06-30.
[*]Script
sesm.shCode: Select all
#!/bin/bash
# sesm.sh Set the Slackware Miirors for usm Repos Bogomips 2017-06-13 19:33:50
# fasm.sh Adaptation by Bogomips to Set Fastest Slackware Repository Mirror 2017-06-07 21:35:10
# Last Update 2017-06-30 21:49:19 Comments; bad(): Output msgs + formatting
Now when looking for a program and come across one dated 2015, I'm reassured if there is a timestamp on page which says last updated 2017. Then know it's the latest version. So similarly, it is that after updating script a day or two may pass before getting time to finish the post. Once post completed, then update the updates entry. Consequently changed title from
Updates to
Updates Posted.
Ed_P wrote:Bogomips wrote:Cut Out can be made from
sesm.sh for the Script
fasm.sh that sets the Fastest Slackware Mirror for a Repo:
Code: Select all
sed "s/\s*sesm\s*().*/fasm \$*/;T;q" $g/sesm.sh > fasm.sh
guest@porteus:~$ bash fasm.sh
Argument usage: <Repository> [<Toggle Debug: 0/1>] [ URL .. ]
URL .. Optional Exclusion Mirror List on Command Line
Optional Exclusion File of URLs: .usm/badmirrors.txt (following format of 'mirrors-<repo>,txt')
Code: Select all
guest@porteus:~$ bash fasm.sh slacky
All Exclusion URLs:
Cmd Line URLs=0 File (.usm/badmirrors.txt) URLs=64 Merged(Unique) URLs=51
Done!
slacky mirror set to: http://slack.isper.sk/pub
Where is the .usm/ folder for the badmirrors.txt file referenced above suppose to be?
This brings us to the matter of choice, and I for one when trying something new or that I'm unfamiliar with do not like having too many choices. Like to be told do this, do this, and Bob's your uncle!
So in this matter the file can be anything you like, just have to edit script accordingly. Taking decision out of hands of user, have decided file should be badmirrors.txt in a directory .usm.
Helpful comments appreciated.
Updated post:
http://forum.porteus.org/viewtopic.php? ... 40e#p55999
BTW ran today with almost no overlap, resulting in much improved time:
Code: Select all
guest@porteus:~$ time sesm all http://slack.isper.sk/pub http://mirror.ovh.net/mirrors/ftp.slackware.com/ http://slackware.mirrorcatalogs.com/ http://mirror.internode.on.net/pub/slackware/
Exclusion URL Specified on Command Line:
http://slack.isper.sk/pub
http://mirror.ovh.net/mirrors/ftp.slackware.com/
http://slackware.mirrorcatalogs.com/
http://mirror.internode.on.net/pub/slackware/
All Exclusion URLs:
Cmd Line URLs=4 File (.usm/badmirrors.txt) URLs=64 Merged(Unique) URLs=67
SLACKWARE
Testing http://mirrors.ucr.ac.cr/slackware/pub/slackware/ 0.616820
Mirrors=152 Processed=32 Remaining=120
Testing ftp://ftp.tu-chemnitz.de/.SAN0/pub/linux/slackware/ 0.676709
Mirrors=152 Processed=46 Remaining=106
Testing ftp://patroklos.noc.ntua.gr/pub/linux/slackware/ 1.573209
Mirrors=152 Processed=56 Remaining=96
Testing http://taper.alienbase.nl/mirrors/slackware/ 0.401148
Mirrors=152 Processed=77 Remaining=75
Testing http://sunsite.icm.edu.pl/packages/linux-slackware/ 0.116343
Mirrors=152 Processed=89 Remaining=63
Testing ftp://ftp.mirrorservice.org/sites/ftp.slackware.com/pub/slackware/ 0.861712
Mirrors=152 Processed=119 Remaining=33
Testing ftp://slackware.mirrors.tds.net/pub/slackware/ 0.614798
Mirrors=152 Processed=145 Remaining=7
Done!
slackware mirror set to: ftp://hpc-mirror.usc.edu/pub/linux/distributions/slackware/
SLACKWAREPATCHES
Done!
slackwarepatches mirror set to: ftp://hpc-mirror.usc.edu/pub/linux/distributions/slackware/
SLACKY
Done!
slacky mirror set to: http://slackware.org.uk/slacky
SALIX
Testing http://download.salixos.org Fail
Mirrors=28 Processed=11 Remaining=17
Testing ftp://ftp.nluug.nl/pub/os/Linux/distr/salix 0.577518
Mirrors=28 Processed=19 Remaining=9
Testing ftp://mirrors.xmission.com/salix 2.207532
Mirrors=28 Processed=25 Remaining=3
Done!
salix mirror set to: http://ftp.nux.ipb.pt/dists/salix
ALIEN
Done!
alien mirror set to: http://bear.alienbase.nl/mirrors/people/alien/sbrepos
PONCE
Done!
ponce mirror set to: http://ponce.cc/slackware
SLACKONLY
Done!
slackonly mirror set to: http://packages.slackonly.com/pub/packages
SBO
Done!
sbo mirror set to: http://slackbuilds.org/slackbuilds
real 1m8.295s
user 0m2.307s
sys 0m1.008s
There is one thing however that is bugging me that only seems to occur with slackware, already happened more than a couple of times, but not with slackware patches: sesm run followed by usm update process
- 1st Update Attempt
Code: Select all
root@porteus:/home/guest# time usm -u all
Starting slackware database update
Downloading: vercheck.txt DONE
Downloading: CHECKSUMS.md5 DONE
Downloading: MANIFEST.bz2 DONE
Downloading: PACKAGES.TXT DONE
Downloading: LIBS.TXT.gz DONE
There was a problem downloading the file:
real 0m4.486s
user 0m0.687s
sys 0m0.919s
guest@porteus:~$ source crm.sh slackware
SLACKWARE
Mirror: ftp://hpc-mirror.usc.edu/pub/linux/distributions/slackware/
151 Commented Out
- 2nd Update Attempt
Code: Select all
guest@porteus:~$ source masm.sh slackware http://mirror.nl.leaseweb.net/slackware
Done!
slackware mirror set to: http://mirror.nl.leaseweb.net/slackware/
root@porteus:/home/guest# time usm -u all
Starting slackware database update
Downloading: vercheck.txt --> ^C
rm: cannot remove '/home/guest/p10/Por/tmp64/usm/vercheck.txt': No such file or directory
real 0m18.573s
user 0m0.534s
sys 0m0.286s
- Finally
Code: Select all
guest@porteus:~$ source masm.sh slackware http://ftp.nluug.nl/os/Linux/distr/slackware
Done!
slackware mirror set to: http://ftp.nluug.nl/os/Linux/distr/slackware/
root@porteus:/home/guest# time usm -u all
Starting slackware database update
...
Starting sbo database update
SLACKBUILDS.TXT.gz
Downloading: SLACKBUILDS.TXT.gz DONE
sbo database updated.
Downloading: extra-deps.txt DONE
Downloading: ignore-deps.txt DONE
real 1m54.067s
user 0m21.265s
sys 0m5.875s
[/list]
Need to make a trite script to Set Alternative Fast Mirror, code name:
safm.sh 8)
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 02 Jul 2017, 02:09
by francois
Is it ripe enough so that you could make a gui to download and run these files?
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 02 Jul 2017, 04:16
by Ed_P
Bogomips wrote:So in this matter the file can be anything you like, just have to edit script accordingly. Taking decision out of hands of user, have decided file should be badmirrors.txt in a directory .usm.
Helpful comments appreciated.
Updated post:
http://forum.porteus.org/viewtopic.php? ... 40e#p55999
Now I understand.
Bogomips wrote:Exclusion File of URLs. As it stands
sesm.sh expects to find the URL list in file:
.usm/badmirrors.txt. Hence
Code: Select all
guest@porteus:~$ [[ -d ,usm ]] || mkdir .usm
For whatever reason I kept thinking the .usm folder was a USM created folder, like /etc/usm/ & /var/usm/, not a user created one. Thanks for clarifying it.
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 02 Jul 2017, 11:20
by Bogomips
francois wrote:Is it ripe enough so that you could make a gui to download and run these files?
Run the scripts pertinent to your situation, and let us know how much riper it has to be.
No idea how to make a gui, so be my guest.
Re: [INFO & SCRIPT] Slackware Mirror Settings for USM
Posted: 03 Jul 2017, 23:27
by Bogomips
Very Final Enhancement: Script to Set Alternative Fast Mirror
safm.sh just about crosses the i's and dots the t's, for me at least.
http://forum.porteus.org/viewtopic.php? ... 6fa#p55999