Ravas coding goodies

[Rava]

I see the backup_loop.py code on the page.

I see that as well.
have a screenshot with the popup in it and the relevant parts marked in red by yours truly:

[Ed_P]

That "secret" applies to this link also: Ravas coding goodies

[Rava]

Tee (command)

In computing, tee is a command in command-line interpreters (shells) using standard streams which reads standard input and writes it to both standard output and one or more files, effectively duplicating its input. It is primarily used in conjunction with pipes and filters. The command is named after the T-splitter used in plumbing.

The tee command might be unfamiliar to some but tee which comes in very handy at times, especially when you have a live system and want to capture outputs (e.g. from /var/log/messages ) even when the system would crash unexpectedly meaning any shutdown / reboot saving-current-settings into a module would fail even when the saving-current-settings script would incorporate saving of /var/log/messages.

Have some examples as well:
https://linuxize.com/post/linux-tee-command/

https://phoenixnap.com/kb/linux-tee


https://unix.stackexchange.com/question ... r-commands

The last line of my script that captures /var/log/messages uses this line:

Code: Select all

tail -fn 10 /var/log/messages|tee -a $myfile

and is meant to be run from a virtual terminal not from your DE so that a mere exit of XWindows would still not interrupt the writing (redirecting) of /var/log/messages into $myfile

Added in 3 minutes 27 seconds:
Of course when the system crashed there would be no writing into $myfile after the crash (duh! that's what a crash is) but at least you would see what was last reported into /var/log/messages - meaning you can examine the reasons of the crash. Unless the system shuts down due to CPU or GPU overheating issues, since this is something the hardware does and it gives no notice whatsoever to your OS.
Unless you write e.g. once per minute the relevant info into /var/log/messages e.g. via the command logger - but that would bloat up your /var/log/messages quite a lot.

Added in 6 minutes 44 seconds:

Code: Select all

sensors|grep Core

gives me the temperature of my CPU cores. I add a line with date/time stamp so that I can compare the recent info with older ones; created a script called mysensors.sh :

Code: Select all

guest@rava:/usr/local/bin$ cat /usr/local/bin/mysensors.sh 
#!/bin/sh
#V0.2
function tz () { 
    echo $(date +%d.%m.%Y\ %H:%M:%S) ____________________________________________________________
}

tz
sensors|grep Core
guest@rava:/usr/local/bin$ mysensors.sh 
13.09.2023 06:36:51 ____________________________________________________________
Core 0:       +63.0°C  (high = +84.0°C, crit = +100.0°C)
Core 1:       +62.0°C  (high = +84.0°C, crit = +100.0°C)
Core 2:       +63.0°C  (high = +84.0°C, crit = +100.0°C)
Core 3:       +62.0°C  (high = +84.0°C, crit = +100.0°C)

[Rava]

updating my daueralarm.sh script

Aside from what I wrote above already, I realized having two sound files chose from (my usual lowered in volume by -15 dB an the non-volume-changed original) can be useful at times, especially when I plan on watching some video (or maybe listening to audio) during the wait time and I want he dauerlalarm sound to be more loud and more annoying )

That means when I wt he script not using the usual approach of traditional syntax and determining what parameter means by the position of the parameter only - I have to put more thought into how to set up that idea.
I think for now i will make a dummy script hat only lists what would be done - interpreting the given parameter (if any) and what result in behaviour the script would think it should have to see if my unorthodox approach of parameter handling is workable at all…

Added in 16 hours 28 minutes 57 seconds:
Ideas for parameter-specific-less calling of daueralarm.sh to turn it into an alarm clock.

no parameter
● standard pause between playing the sound (2 seconds)
● standard mp3 file (-15dB)
● no pause prior executing the sound-loop
when useful? e.g.when you have a longer program from the CLI to run (e.g. new encoding of a longer video) to be alerted by audio as soon as it is finished; you can add a visual alert via

Code: Select all

notify-send -i info -t 0 "Encoding" "finished"

one parameter
● parameter sets the pause in seconds between playing the sound
● standard mp3 file (-15dB)
● no pause prior executing the sound-loop
same reason as above: to be alerted when a longer process is finished.

two parameters:
● 1st parameter sets the pause in seconds between playing the sound
● 2nd parameter : s = standard mp3 file (-15dB); l = louder mp3 file (0dB)
● no pause prior executing the sound-loop


three parameters:
● 1st parameter sets the pause in seconds between playing the sound
● 2nd parameter : s = standard mp3 file (-15dB); l = louder mp3 file (0dB)
● 3rd parameter: pause prior executing the sound-loop (in the way pause accepts the pause-parameter: 70 = 70 seconds; 2m = 2 minutes; 2.1m = 2 minutes 6 seconds etcetera.
When you want to use it as self-coded alarm clock (e.g. to set it for 9m to get the alarm when the pasta is cooked)

[Rava]

Lazy me quoting beny and myself:

Hi roro manual login and startx Is Better,well on my system try it

[Porteus supports the "3" kernel cheat code, add it to our kernel append parameter and you boot into init 3 (Slackware standard, Virtual Text terminals, multiuser, network)]

I prefer it that way as well, it takes a mere seconds to manually log in as guest and then type

Code: Select all

startx

but gives you much more control, also you can scroll up in the 1st terminal and review all boot messages even when they are already scrolled outside of the visible screen, something you cannot do when you boot into your XWindows and your chosen DE.

Added in 3 minutes 59 seconds:
Cave! The scrolling up only works when you did not change to a different virtual terminal ("VT"). When you stay in one virtual terminal you can scroll up to see much more output of current program or from boot, but when you change the VT you can only access what is currently seen, all that is scrolled up is gone.

for those who do not know, scrolling up or scrolling down in a VT is done via

Code: Select all

Shift + PageUp

or

Code: Select all

Shift + PageDown

Thanks beny.

Added in 2 hours 25 minutes 44 seconds:
And one more quoting, this time Ed_P and me. It's about how to efficiently use mc navigating around the file system using uniquely named symlinks.

But I don't see the /tmp folder in it.

⬤ When using mc you can either navigate through the folders like usual (double-click or press return on a folder, move up by doing that on the ".." entry)

⬤ or just type in the command prompt below the command

Code: Select all

cd /tmp

⬤ or start mc with the two folders you want to copy files in between.
Let's assume your module is in /tmp/ and you have a symlink that links to your main Porteus modules folder called /Porteus_modules
then just type

Code: Select all

mc  /tmp/ /Porteus_modules

Usually it would be enough to type

Code: Select all

mc  /tmp/ /P[tab]

since by default there exists no folder or file in / that starts with a upper-case P like your link /Porteus_modules does.
The [tab] then expands it to the only existing name -> /Porteus_modules

Yes, in some many cases using the terminal (and that includes mc) can let you do things quicker than a GUI equivalent especially when you use symlinks named in an easily remembered and unique way (as in: the first character of your symlink doesn't exist in that very folder aside from your symlink)

[Rava]

How To set up a small logging script that - like syslog - keeps logging for all eternity - only stops when you reboot, the system crashes or you manually zero the target file (for when the old logged info is no longer needed and you want to conserve space on that drive) or you exit the script forcefully via kill or Ctrl+C

You have to use a physical drive for all of it to make sense. You could run it in your aufs and hope your system will never crash and your auto-save-module on exit will also save your log file. But that is not how you should do things. When you set up any kind of log you will have a reason for doing so (e.g. your system crashed because of several possible reasons, and you alter my below script to monitor the possible reasons)

My example is quite easy.

You create a script, best in an ext[234] partition, in a folder, not in the root of that partition.

Let's presume that folder is /mnt/sda3/log/ and your script is called mysensorslog.sh and sits in /mnt/sda3/log/mysensorslog.sh

Here the code:

Code: Select all

#!/bin/sh
#based on mysensors.sh V0.2 and tail+tee_VLM.sh V0.2
function tz () { 
    echo $(date +%d.%m.%Y\ %H:%M:%S) ____________________________________________________________
}
if [ $UID -ne 0 ]; then
    echo "$0: ERROR - start me as user root. Aborting."
    exit 1
fi
mydate=$(date +%Y-%m-%d)
echo mydate suffix I am using: $mydate
myfile=sensors_$mydate
echo myfile I am using: $myfile - WILL OVERWRITE EXISTING ENTRIES
echo $(date +%d.%m.%Y\ %H:%M:sensors_%S) started: $0 >$myfile
while true; do { tz|tee -a $myfile; sensors|grep Core|tee -a $myfile ; sleep 1m ; } done

What does it do?
It sets up its file name based on the current date - sensors_YYYY--MM-DD

Here is when I run it as example:

Code: Select all

guest@rava:/mnt/sda3/log$ ls -o sensors_2023-09-20 
-rw-r--r-- 1 root 18618 2023-09-20 02:20 sensors_2023-09-20

This is what the script tells you when first started:

Code: Select all

root@rava:/mnt/sda3/log# ./mysensorslog.sh 
mydate suffix I am using: 2023-09-20
myfile I am using: sensors_2023-09-20 - WILL OVERWRITE EXISTING ENTRIES
20.09.2023 01:20:56 ____________________________________________________________
Core 0:       +56.0°C  (high = +84.0°C, crit = +100.0°C)
Core 1:       +55.0°C  (high = +84.0°C, crit = +100.0°C)
Core 2:       +57.0°C  (high = +84.0°C, crit = +100.0°C)
Core 3:       +55.0°C  (high = +84.0°C, crit = +100.0°C)

Every minute a new set of log info will be displayed on the terminal that runs the script and also written into its log file (in my example called sensors_2023-09-20 )

Code: Select all

20.09.2023 01:20:56 ____________________________________________________________
Core 0:       +56.0°C  (high = +84.0°C, crit = +100.0°C)
Core 1:       +55.0°C  (high = +84.0°C, crit = +100.0°C)
Core 2:       +57.0°C  (high = +84.0°C, crit = +100.0°C)
Core 3:       +55.0°C  (high = +84.0°C, crit = +100.0°C)
20.09.2023 01:21:56 ____________________________________________________________
Core 0:       +55.0°C  (high = +84.0°C, crit = +100.0°C)
Core 1:       +52.0°C  (high = +84.0°C, crit = +100.0°C)
Core 2:       +56.0°C  (high = +84.0°C, crit = +100.0°C)
Core 3:       +52.0°C  (high = +84.0°C, crit = +100.0°C)
20.09.2023 01:22:56 ____________________________________________________________
Core 0:       +55.0°C  (high = +84.0°C, crit = +100.0°C)
Core 1:       +52.0°C  (high = +84.0°C, crit = +100.0°C)
Core 2:       +55.0°C  (high = +84.0°C, crit = +100.0°C)
Core 3:       +52.0°C  (high = +84.0°C, crit = +100.0°C)

You can change the pause between displaying and writing info to the log file to any period you want, e.g. every 30 seconds, every minute (my example), every 5 minutes, every 10 minutes, every hour…


And my example being a mere log about the CPU core temperatures is also just an easy example. Use any command you want to be logged, just start with this part

Code: Select all

tz|tee -a $myfile;

since a log without info when something you logged started going off the rails (in this example: the temperature logged is approaching the critical temperature) is of not much help afterwards when your system did fail and you examine the log file (examining sensors_2023-09-20 in my example)

When you plan on leaving the machine on its own and you made sure the error you are monitoring for did not happen up to the current time you can erase the contents of the log file from another terminal.
Open a new tab of your terminal program and log in as root an do this:

Code: Select all

echo -n >/mnt/sda3/log/sensors_2023-09-20

When you would monitor the log file via a

Code: Select all

tail -f /mnt/sda3/log/sensors_2023-09-20

that tail will report this:

Code: Select all

tail: /mnt/sda3/log/sensors_2023-09-20: file truncated

But since the terminal that runs the script does display what the program reports and writes the output into the log file, you do not need to monitor the log via tail -f /mnt/sda3/log/sensors_2023-09-20


When you want to be extra secure do this

Append "3" to your Porteus kernel line at boot time

Your DE will not start, instead you will see the Virtual Terminal ("VT")
Log into the 1st VT as root, move to the correct folder and execute the script like so:

Code: Select all

root@rava:~# cd  /mnt/sda3/log/
root@rava:/mnt/sda3/log/# ./mysensorslog.sh 

Then switch to another VT, log in as guest and start X via

Code: Select all

startx

Your DE will start, but in your 1st VT your root script will continue to log .

And here it can be handy to monitor the log in your DE since you do not want to switch from your Xwindows session to the VT and back to Xwindows to just see the last entries of the log. Therefore a

Code: Select all

tail -f /mnt/sda3/log/sensors_2023-09-20

started from a Xwindows terminal Emulator would be a good idea.

I hope I was able to explain all details good enough so you understand the workings and can alter my script to fit your needs.

Added in 13 minutes 16 seconds:
Cave! Do not alter the script without any valid sleep command at the end of the while true loop.
I recommend not going any lower than logging every 10 seconds since every log strains your system, and remember: you have your DE and all your usual programs running as well, you do not want a mere log to go berserk in the background (filling up your partition that has the log file on it and also using too much IO / write cycles - e.g. when you only have a mere 1 second pause and log much more than my 4 lines of info and the one line with the date/time)

Please understand what your script does prior changing any script.

[Rava]

Only with the help of the wonderful people at https://stackoverflow.com my inept self was able to get a working code for creating a script that would execute make-ffplay-script for each folder directly in the current folder thus creating for each FOLDER a FOLDER.sh that will use ffplay for playing multimedia files that sit in FOLDER/ or in FOLDER/ and FOLDER/SUBFOLDER/

Cave! Even when the current version is 3.50, my make-ffplay-script is still too stupid to check if there are actual multimedia files in the folder it was given as parameter. It will create is FOLDERNAME.sh regardless and executing that script will do… nothing.
(You find my make-ffplay-script somewhere above)

Special thanks to user pjh @ stackoverflow

My preferred approach uses printf:

Code: Select all

for d in */ ; do printf 'make-ffplay-script %q\n' "${d%/}"; done > myfindloop_pjh.sh
chmod -v a+x myfindloop_pjh.sh

But echo with added " also works:

Code: Select all

for d in */ ; do echo make-ffplay-script \""$d"\"; done > myfindloop.sh
chmod -v a+x myfindloop.sh

[Rava]

Got tipped towards this https://mywiki.wooledge.org/BashFAQ/020 and it is just so good I have to share it. All of https://mywiki.wooledge.org/ is a read worthwhile when you are interested in the working of the shell and about script coding.

How can I find and safely handle file names containing newlines, spaces or both?

First and foremost, to understand why you're having trouble, read Arguments to get a grasp on how the shell understands the statements you give it. It is vital that you grasp this matter well if you're going to be doing anything with the shell.

The preferred method to deal with arbitrary filenames is still to use find(1):

Code: Select all

find ... -exec command {} \;

or, if you need to handle filenames en masse:

Code: Select all

find ... -exec command {} +

xargs is rarely ever more useful than the above, but if you really insist, remember to use -0 (-0 is not in the POSIX standard, but is implemented by GNU and BSD systems):

Code: Select all

# Requires GNU/BSD find and xargs
find ... -print0 | xargs -r0 command

# Never use xargs without -0 or similar extensions!

Use one of these unless you really can't.

Another way to deal with files with spaces in their names is to use the shell's filename expansion (globbing). This has the disadvantage of not working recursively (except with zsh's extensions or bash 4's globstar), and it normally does not include hidden files (filenames beginning with "."). But if you just need to process all the files in a single directory, and omitting hidden files is okay, it works fantastically well.

For example, this code renames all the *.mp3 files in the current directory to use underscores in their names instead of spaces (this uses the bash/ksh extension allowing "/" in parameter expansion):

Code: Select all

# Bash/ksh
for file in ./*\ *.mp3; do
  if [ -e "$file" ] ; then  # Make sure it isn't an empty match
    mv "$file" "${file// /_}"
  fi
done

You can omit the "if..." and "fi" lines if you're certain that at least one path will match the glob. The problem is that if the glob doesn't match, instead of looping 0 times (as you might expect), the loop will execute once with the unexpanded pattern (which is usually not what you want). You can also use the bash extension "shopt -s nullglob" to make empty globs expand to nothing, and then again you can omit the if and fi.

For more examples of renaming files, see FAQ #30.

Remember, you need to quote all your Parameter Expansions using double quotes. If you don't, the expansion will undergo WordSplitting (see also argument splitting and BashPitfalls). Also, always prefix globs with "/" or "./"; otherwise, if there's a file with "-" as the first character, the expansions might be misinterpreted as options.

Another way to handle filenames recursively involves using the -print0 option of find (a GNU/BSD extension), together with bash's -d extended option for read:

Code: Select all

# Bash
unset a i
while IFS= read -r -d $'\0' file; do
  a[i++]="$file"        # or however you want to process each file
done < <(find /tmp -type f -print0)

The preceding example reads all the files under /tmp (recursively) into an array, even if they have newlines or other whitespace in their names, by forcing read to use the NUL byte (\0) as its line delimiter. Since NUL is not a valid byte in Unix filenames, this is the safest approach besides using find -exec. IFS= is required to avoid trimming leading/trailing whitespace, and -r is needed to avoid backslash processing. In fact, $'\0' is actually the empty string (bash doesn't support passing NUL bytes to commands even built-in ones) so we could also write it like this:

Code: Select all

# Bash
unset a i
while IFS= read -r -d '' file; do
  a[i++]="$file"
done < <(find /tmp -type f -print0)

So, why doesn't this work?

Code: Select all

# DOES NOT WORK
unset a i
find /tmp -type f -print0 | while IFS= read -r -d '' file; do
  a[i++]="$file"
done

Because of the pipeline, the entire while loop is executed in a SubShell and therefore the array assignments will be lost after the loop terminates. (For more details about this, see FAQ #24.)

For a longer discussion about handling filenames in shell, see Filenames and Pathnames in Shell: How to do it Correctly.

CategoryShell

Other essential reads:
https://mywiki.wooledge.org/BashProgramming
https://mywiki.wooledge.org/BashPitfalls