Ravas coding goodies

For discussions about programming and projects not necessarily associated with Porteus.
User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#166 by Rava » 31 Jan 2023, 21:39

Here my skeleton .wget-nameloop-double.sh
There is also a .wget-nameloop-single.sh The difference: one uses single quotes for the filenames, the other double quotes.
Single quotes are more robust, but will fail when the $namebase contains single quotes as well.
Like nameext="I've gotten yadda yadda " will work, when

Code: Select all

I've gotten yadda yadda 
is the title, but
nameext='I've gotten yadda yadda ' will fail.

Code: Select all

#!/bin/bash
#V0.3 - added || notify-send -i error * $sourceurl
namebase="TITLE "
nameext=".mp4"
filename=wget.sh
sourceurl=
declare -i n=12
declare -i i

echo '#!/bin/bash' >> $filename
echo '#	'$sourceurl >> $filename
echo -n "processing # "
for ((i = 1; i <= ${n}; i++)); do  { 
	if [ $i -lt 10 ]; then
		number=0${i}
	else
		number=${i}
	fi
	echo '#wget -c '\'\'' -O "'$namebase$number$nameext'" && notify-send -i info -t 0 "Downloading '$number'" "finished" || notify-send -i error -t 0 "Downloading '$number'" "failed"'  >> $filename
	echo -n $number" "
} done 
echo "echo press enter
read" >> $filename
chmod u+x $filename
Replace "TITLE " with the title.
Set the sourceurl= if you want, you can keep it blank as well.

As example, using "TEST " as $namebase and declare -i n=2 and sourceurl=http://example.net/TEST the script produces this wget.sh script and makes it executable:

Code: Select all

#!/bin/bash
#	http://example.net/TEST
#wget -c '' -O "TEST 01.mp4" && notify-send -i info -t 0 "Downloading 01" "finished" || notify-send -i error -t 0 "Downloading 01" "failed"
#wget -c '' -O "TEST 02.mp4" && notify-send -i info -t 0 "Downloading 02" "finished" || notify-send -i error -t 0 "Downloading 02" "failed"
echo press enter
read
Of course, the more single episodes you are planning of downloading, the more help the script gives you.

Why are the

Code: Select all

#wget -c 
lines commented out by default?
Because it is meant for you to manually hunt down the URLs for each episode first.
When you have one URL, uncomment the line and insert it at the correct place.
E.g. like so:

Code: Select all

#!/bin/bash
#	http://example.net/TEST
wget -c 'http://example.net/random1/TEST01.mp4' -O "TEST 01.mp4" && notify-send -i info -t 0 "Downloading 01" "finished" || notify-send -i error -t 0 "Downloading 01" "failed"
wget -c 'http://example.net/random2/TEST02.mp4' -O "TEST 02.mp4" && notify-send -i info -t 0 "Downloading 02" "finished" || notify-send -i error -t 0 "Downloading 02" "failed"
echo press enter
read
When you happen to have all URLs in advance and they are something like this

Code: Select all

http://example.net/TEST01.mp4
http://example.net/TEST02.mp4
[…]
http://example.net/TEST24.mp4
then my above script is less useful, then you can tweak the script to include the target URLs as well.

Have a screenshot from yesterday :
Image
:D
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#167 by Rava » 29 Aug 2023, 23:26

In case it is unknown to you:

https://virusscan.jotti.org/

This is by far the best online virus malware trojan scanner - because it uses 14 scan engines, many of these well known ones.
About Jotti's malware scan
Jotti's malware scan is a free service that lets you scan suspicious files with several anti-virus programs. You can submit up to 5 files at the same time. There is a 250MB limit per file. Please be aware that no security solution offers 100% protection, not even when it uses several anti-virus engines. All files are shared with anti-virus companies so detection accuracy of their anti-virus products can be improved.
Added in 1 day 7 hours 42 minutes 16 seconds:
I am planning on updating my daueralarm.sh script, but I do not want it to use standard parameters (e.g. -s [sleeptime] -p [pause between playing the scheduled sound file]), instead it should be determined by the position of the parameter what it's about.

When using it with sleep to simulate an alarm clock function I use this one liner:

Code: Select all

tz;sleep 30m;tz;daueralarm.sh 15
tz (trennline zeit = divider-line time) displays just that: current date/time and a divider line to make sections of output more easy to read, e.g. like so:

Code: Select all

root@porteus:~# tz
31.08.2023 09:08:02 ____________________________________________________________
When I forgot that and just use

Code: Select all

sleep 30m;daueralarm.sh 15
there is the issue that I do not know when I started that one-liner. It could be 10 minutes ago or 29 minutes and 50 seconds - no quick and easy way to tell.

Therefore I plan on incorporating the sleep function into daueralarm.sh - and if you use that sleep function before playing the sound loop, it will also display the equivalent of tz.

If anyone has an idea how to improve my rough first design idea, please tell me.
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#168 by Rava » 06 Sep 2023, 02:43

Like I wrote above, I would have taken considerations and ideas from you folks into account.

But since not a single one came up with any suggestion or idea - therefore I presume no one is really interested in my alarm-clock script - I just do whatever I please instead.
Currently it's like so:

Code: Select all

guest@porteus:~$ tz;sleep 2m;tz;daueralarm.sh 15
06.09.2023 04:06:47 ____________________________________________________________
06.09.2023 04:08:47 ____________________________________________________________
daueralarm.sh V1.2 - Sounddatei ist /mnt/sda4/sound/notification_-15dB.mp3
Pausenlaenge: 15 Sekunden - Abbruch mit Strg+C
^C
guest@porteus:~$ tz;sleep 45;tz;daueralarm.sh 15
06.09.2023 04:33:07 ____________________________________________________________
06.09.2023 04:33:52 ____________________________________________________________
daueralarm.sh V1.2 - Sounddatei ist /mnt/sda4/sound/notification_-15dB.mp3
Pausenlaenge: 15 Sekunden - Abbruch mit Strg+C
^C
guest@porteus:~$ 
I would like it to me more like so:

Code: Select all

guest@porteus:~$ daueralarm.sh 15 7m
daueralarm.sh V1.3 - Schlafe fuer: 7m - Pausenlaenge: 15 Sekunden
06.09.2023 04:36:44 ____________________________________________________________
06.09.2023 04:43:44 ____________________________________________________________
Pausenlaenge: 15 Sekunden - Abbruch mit Strg+C
^C
guest@porteus:~$ 
And when only given one parameter it is the pause between playing the soundfile (no sleep; this mode is needed when I want it to alert me when any longer-taking terminal task is finished.). And with no parameter it is its default pause between playing the soundfile (= 2 seconds).
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#169 by Rava » 10 Sep 2023, 09:17

While I am too lazy to actually work on improving my daueralarm.sh as described above, I was looking into various ways to debug a script and thought I could share some of my finds here.

https://stackoverflow.com/questions/951 ... ash-script
sh -x script [arg1 ...]
bash -x script [arg1 ...]

These give you a trace of what is being executed. (See also 'Clarification' near the bottom of the answer.)

Sometimes, you need to control the debugging within the script. In that case, as Cheeto reminded me, you can use:

set -x

This turns debugging on. You can then turn it off again with:

set +x

(You can find out the current tracing state by analyzing $-, the current flags, for x.)
The advantage of using bash -x script instead of adding set -x is that you do not need to edit the script, especially helpful when you want to debug it as user gust but it is a system script not writeable for anyone but root.

And you can use https://www.shellcheck.net/ to check our script.

Also CAVE! In Porteus, the command sh is a symlink to bash but do not expect it to be like this on every system. When you learn to bash-script on Porteus you might presume sh is a symlink to bash on every Linux system out there, and you could run into troubles when you are working on a system where sh is a more restricted shell of its own, e.g. sh could be a link to dash shell o these systems, so do not expect bash specifics to work universally on all sh shells in all Linux environments. If yu do you will at some point create unneeded headaches for you when ou get to a system that uses e.g. dash as target for the symlink sh.
Or even in Porteus you can get intousues when the script you created is meant to work in the boot debug environment (that pops up when the kernel cannot find the needed Porteus module files) - that is a very restricted shell and most of the bash stuff will not work, even when a command is known it is just a symlink to its main binary with reduced valid arguments.
It is of course BusyBox I am talking about here. :)

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#170 by Rava » 10 Sep 2023, 09:21

I also thought https://betterprogramming.pub/my-minima ... 0759114040 would be helpful, but it is less so.
The author of that article made it so that you need a medium.com account to actually read the article. :wall:
Otherwise "minimal-safe-bash-script-template" would be a good article to share.

Added in 5 minutes 18 seconds:
And while bashscripts.org is no longer on the internet, you still might find quite some valuable stuff via web archive like so: https://web.archive.org/web/20080509125 ... ripts.org/
Some quotes from that
General suggestions and the VERY basics.
The Hello world script, Commenting / Documentation, echo command, vi, emacs, other text editor comments, shell command or program?

Aliases and Dotfiles
For .bashrc information and for posting your interesting/useful aliases. Also post other dotfile info here.

Sed & Awk
sed & awk, two text manipulation programs that are mainstays of the UNIX programmer's toolbox.
Be aware that the link is one specific snapshot of that site. Older or newer snapshots will differ, some might be more helpful, some might be less so.
20080509125854 in the URL of archive.org means: YYYYMMDDHHMMSS
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Ravas coding goodies

Post#171 by Ed_P » 10 Sep 2023, 13:40

Certainly an interesting script.
Rava wrote:
10 Sep 2023, 09:26
The author of that article made it so that you need a medium.com account to actually read the article.
Otherwise "minimal-safe-bash-script-template" would be a good article to share.
The Safe script is easy enough to see and copy using the view raw option. The article says the full story can be read with a free Medium account, maybe you could create an account that you share with other interested Safe parties.
Rava wrote:
10 Sep 2023, 09:26
20080509125854 in the URL of archive.org means: YYYYMMDDHHMMSS
Interesting. How did you determine that?
Ed

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#172 by Rava » 10 Sep 2023, 13:47

Ed_P wrote:
10 Sep 2023, 13:40
The article says the full story can be read with a free Medium account, maybe you could create an account that you share with other interested Safe parties.
I think you could run into copyright issues when you get access to some stuff and share that with others who not have the same access.
in this case, they would use m email and other collected data to make money from me, while folks I would share the info with would not have given medium anything.

But I did not see the "view raw option" - (could be me using Palemoon or me using the Advanced Night Mode - i even changed the ANM settings to several of its options but still saw no "view raw" option.) I looked into the html source code and it was random data displayed where the script should have been.

But I am sure when one gets the raw script text one would find some very similar "minimal-safe-bash-script-templates" on the web that are free to access.
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Ravas coding goodies

Post#173 by Ed_P » 10 Sep 2023, 14:40

Rava wrote:
10 Sep 2023, 13:47
But I did not see the "view raw option"
It's at the bottom of the script and on the right of the "template.sh hosted with ❤ by GitHub" line.
Ed

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#174 by Rava » 10 Sep 2023, 14:42

Ed_P wrote:
10 Sep 2023, 14:40
"template.sh hosted with ❤ by GitHub" line.
Then it should be available on github as well, and quite freely available.
Did you by any change find it on some github URL?
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Ravas coding goodies

Post#175 by Ed_P » 10 Sep 2023, 15:19

Rava wrote:
10 Sep 2023, 14:42
Did you by any change find it on some github URL?
The "view raw" option takes one to here: https://gist.githubusercontent.com/Bett ... emplate.sh which is a part of https://gist.githubusercontent.com/BetterProgramming/.

FYI I wasn't able to post this in Porteus 5.0, I had to switch to Windows 11 to do it. The form was invalid.
Ed

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#176 by Rava » 10 Sep 2023, 16:14

Ed_P wrote:
10 Sep 2023, 15:19
The "view raw" option takes one to here: https://gist.githubusercontent.com/Bett ... emplate.sh which is a part of https://gist.githubusercontent.com/BetterProgramming/.
Thanks.
https://gist.githubusercontent.com/BetterProgramming/ states
Better Programming is a Medium publication devoted to bringing you daily engineering content to help you become a better programmer!
Highlighting by me.
"daily engineering content" when the last post on it was November 9, 2022 19:12 :crazy:

Now once could use the date -d @nnnnn way to convert that date/time into unix time (=seconds till 1.1.1970) and convert the current day/time into unix time, subtract each other and convert the resulting difference in seconds using this:

Code: Select all

function seconds2time () 
{ 
    local T=$1;
    local D=$((T/60/60/24));
    local H=$((T/60/60%24));
    local M=$((T/60%60));
    local S=$((T%60));
    if [[ ${D} != 0 ]]; then
        if [[ ${D} = 1 ]]; then
            printf '%d day %02d:%02d:%02d' $D $H $M $S;
        else
            printf '%d days %02d:%02d:%02d' $D $H $M $S;
        fi;
    else
        printf '%02d:%02d:%02d' $H $M $S;
    fi
}
:D

but that would be being picky so I refrain doing that. :D :teehee:
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#177 by Rava » 10 Sep 2023, 16:16

But now we can freely post the template here:
source: https://gist.githubusercontent.com/Bett ... emplate.sh

Code: Select all

#!/usr/bin/env bash
#: Your comments here.
set -o errexit
set -o nounset
set -o pipefail
work_dir=$(dirname "$(readlink --canonicalize-existing "${0}" 2> /dev/null)")
readonly conf_file="${work_dir}/script.conf"
readonly error_reading_conf_file=80
readonly error_parsing_options=81
readonly script_name="${0##*/}"
a_option_flag=0
abc_option_flag=0
flag_option_flag=0
trap clean_up ERR EXIT SIGINT SIGTERM
usage() {
    cat <<USAGE_TEXT
Usage: ${script_name} [-h | --help] [-a <ARG>] [--abc <ARG>] [-f | --flag]
DESCRIPTION
    Your description here.
OPTIONS:
-h, --help
        Print this help and exit.
-f, --flag
        Description for flag option.
-a
        Description for the -a option.
--abc
        Description for the --abc option.
USAGE_TEXT
}
clean_up() {
    trap - ERR EXIT SIGINT SIGTERM
    # Remove temporary files/directories, log files or rollback changes.
}
die() {
    local -r msg="${1}"
    local -r code="${2:-90}"
    echo "${msg}" >&2
    exit "${code}"
}
if [[ ! -f "${conf_file}" ]]; then
    die "error reading configuration file: ${conf_file}" "${error_reading_conf_file}"
fi
# shellcheck source=script.conf
. "${conf_file}"
parse_user_options() {
    local -r args=("${@}")
    local opts
# The following code works perfectly for 
    opts=$(getopt --options a:,f,h --long abc:,help,flag -- "${args[@]}" 2> /dev/null) || {
        usage
        die "error: parsing options" "${error_parsing_options}"
    }
eval set -- "${opts}"
while true; do
    case "${1}" in
--abc)
            abc_option_flag=1
            readonly abc_arg="${2}"
            shift
            shift
            ;;
-a)
            a_option_flag=1
            readonly a_arg="${2}"
            shift
            shift
            ;;
--help|-h)
            usage
exit 0
            shift
            ;;
--flag|-f)
            flag_option_flag=1
shift
            ;;
--)
            shift
            break
            ;;
        *)
            break
            ;;
    esac
    done
}
parse_user_options "${@}"
if ((flag_option_flag)); then
    echo "flag option set"
fi
if ((abc_option_flag)); then            # Check if the flag options are set or ON:
    # Logic for when --abc is set.
    # "${abc_arg}" should also be set.
    echo "Using --abc option -> arg: [${abc_arg}]"
fi
if ((a_option_flag)); then
    # Logic for when -a is set.
    # "${a_arg}" should also be set.
    echo "Using -a option -> arg: [${a_arg}]"
fi
exit 0
I called it bash-template.sh and not just template.sh though.

Added in 47 minutes 35 seconds:
And don't forget, when you open a *.sh file with an editor that supports source code highlighting the code parts will be displayed in several colours based on what they are, code-logic-wise.
E.g. mousepad or geany do support code highlighting, :)

and that makes reading, editing and debugging of any code much easier. :celebrate3:
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Ravas coding goodies

Post#178 by Ed_P » 10 Sep 2023, 20:32

Rava wrote:
10 Sep 2023, 17:04
I called it bash-template.sh and not just template.sh though.
safe-template.sh would be more appropriate. imo
Ed

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Ravas coding goodies

Post#179 by Rava » 10 Sep 2023, 23:52

Ed_P wrote:
10 Sep 2023, 20:32
safe-template.sh would be more appropriate
A template worth its salt should be safe by default.
But since the ending of .sh stands for any kind of more-or-less sh compatible shell, where is the info in your name that it is specifically a bash script?

At least call it bash-safe-template.sh or safe-bash-template.sh :D

Added in 11 minutes 8 seconds:
I wondered why do I get no result searching for template.sh on BetterProgramming?
https://github.com/search?q=owner%3ABet ... positories
0 results (104 ms) in BetterProgramming
I presume the reason for that is that this gist is set to secret.

Example: https://gist.github.com/gecheline/39f48 ... 7856354fda
When your mouse hovers over "secret" this popup comes up: "Only those with the link can see this gist." :)

it is one part of an attempt to recover data from a corrupt (physically damaged it seems) external hard drive. https://betterprogramming.pub/corrupted ... eb2a9e0422
An actual estimate of the remaining backup time using Disk Drill. For context, 8489 hours is 354 days, or a little less than A YEAR.
[…]
This simple Python-based approach managed to save 78% of the important data on this corrupted external drive when fancy apps failed to even recover a single byte. Time-wise, writing up the script took less than half an hour, troubleshooting it another hour or so, and the backup itself took about two days (if this sounds too long, remember the 8,000+ hours Disk Drill estimated!). Depending on the size of the backup and your patience, this can even be further optimized with multiprocessing.

And, the best part — it’s free!
:)

Added in 14 minutes 41 seconds:
Image
A visualization of the parts recovered vs lost across the chosen “important” directories.
That's one neat python script right there.
I will now go offer a sacrifice to the Python gods.
:ROFL: ImageImage
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Ravas coding goodies

Post#180 by Ed_P » 11 Sep 2023, 05:57

Rava wrote:
11 Sep 2023, 00:41
since the ending of .sh stands for any kind of more-or-less sh compatible shell, where is the info in your name that it is specifically a bash script?
SafeBash.sh works for me.
Rava wrote:
11 Sep 2023, 00:41
I wondered why do I get no result searching for template.sh on BetterProgramming?
Did you try the page's advanced search option?
Rava wrote:
11 Sep 2023, 00:41
Example: https://gist.github.com/gecheline/39f48 ... 7856354fda
When your mouse hovers over "secret" this popup comes up: "Only those with the link can see this gist."
I see the backup_loop.py code on the page.
Ed

Post Reply