Ravas coding goodies

For discussions about programming and projects not necessarily associated with Porteus.
User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#16 by Rava » 02 Nov 2020, 10:11

Ed_P wrote:
02 Nov 2020, 06:28
I don't understand your echo code. Initially the bold and yellow colors are coded as: ${color} but then the off and yellow are specified without the brackets: $off$yel. Does this work? I thought the brackets were mandatory.
When you want to be on the safe side, and also code in a style that it outright the same, using brackets all the time would be your style.

But when you just have the $off at the end of a text output, a bracket is not needed. It is needed when it would create ambiguity for the shell, and then it is truly mandatory, in all other cases it is good style but voluntary.

I just code my stuff at it comes and know by case by case and line by line what is needed, and when I can get along without issues without brackets.

HTH.
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#17 by Rava » 10 Nov 2020, 07:07

moved some explanation here since it was off topic in the original thread:
________________________________________________________
Ed_P wrote:
17 Dec 2018, 05:38
In Cinnamon $PWD doesn't point to /tmp.
I thought PWD means "Present Working Directory", but I could be wrong, that is how I read it and it is usually just what $PWD does.
Change your bash shell into some random folder and use $PWD to ask where you are, like so:

Code: Select all

guest@porteus:~$ cd
guest@porteus:~$ echo $PWD
/home/guest
guest@porteus:~$ cd /tmp/
guest@porteus:/tmp$ echo $PWD
/tmp
guest@porteus:/tmp$ cd /etc/UPower/
guest@porteus:/etc/UPower$ echo $PWD
/etc/UPower
HTH!
____________________________________
Ed_P wrote:
17 Dec 2018, 05:38
I've never heard of "." to refer to a directory. "/." yes.
Could it be you confused "./" with "/."?
Cause /. refers to the root folder:

Code: Select all

guest@porteus:/etc/UPower$ ls /.
etc   lib64  opt   run    srv  usr    bin     home  media  proc  sbin   sys  var   dev     lib   mnt    root  tmp 
guest@porteus:/etc/UPower$ ls ./
UPower.conf
Ed_P wrote:
17 Dec 2018, 05:38
I've never heard of "." to refer to a directory.
Just try it like so:

Code: Select all

cp /path/filename .
You won't get a filename named ".", instead /path/filename will be copied into the directory the shell is in, aka "$PWD" aka "."

BTW, the "." in the directory listing stands for "current directory" and the ".." stands for "one directory up":

Code: Select all

guest@porteus:/etc/UPower$ ls -a
.  ..  UPower.conf
In this example "." is /etc/UPower and ".." is /etc

The "." and the ".." is similar to $PWD, it is not a fixed variable, but it changes accordingly to the folder the current shell is in.

Also, quoted from here http://www.linfo.org/current_directory.html
A single dot is also used to represent the current directory in some commands, scripts (i.e., short programs) and the PATH environmental variable.
Compare guest's $PATH with root's $PATH:

Code: Select all

guest@porteus:/etc/UPower$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/games:/opt/porteus-scripts:.
vs

Code: Select all

root@porteus:~# echo $PATH 
/usr/local/sbin:/usr/local/bin:/sbin:/usr/sbin:/bin:/usr/bin
The missing "." for root means it won't execute programs or scripts from the current directory, but in guest's case it will do so.
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 5795
Joined: 06 Feb 2013, 22:12
Distribution: 4.0 & 5.0rc2 Cinnamon 64 ISOs
Location: Western NY, USA

Ravas coding goodies

Post#18 by Ed_P » 10 Nov 2020, 17:33

Thank you Rava. :beer:

How did you learn bash scripting so well?
Ed

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#19 by Rava » 10 Nov 2020, 17:36

^ by reading free bash scripting tutorials / bash scripting books. And reading man pages.

______________________________________

from here: User advice needed: welcome window at first boot for ver. 5.0 (Post by Rava #79540)

I change my root password in the running system via script to be run as root - readable only to root (security reasons) - like so

Code: Select all

#!/bin/bash
VERSION="0.1"
MYNAME=mypasswd-reset
echo -e $yel$bld$MYNAME$off$yel V$VERSION$off
echo 'root:$5$ta0pL/AtY.S/B$o8ua3In1IRPSRbmWpjp4CXgsgTzDJ/UY2Kt/7dYxbGD' | chpasswd -e
this is the variant to change password back to "root". Where do you get the seemingly random characters from? look into /etc/shadow

And after changing root password into a strong one by running just one script that holds the password never verbatim (after logging in as root with password toor at least once) I never encountered any issues with any porteus scripts or modules. :D

Additional info
1. look into /etc/shadow to see which parts you need for the script when root password is still toor
1. Change root password to a strong one
2. look into /etc/shadow - copy the encrypted part into echo 'root:XXX' | chpasswd -e replacing the XXX
3. Put the line into a script readable and executable only for root, like in the example above.

4. to be safe: also create a script with the above code so you can switch the root password back to toor. Of course you need at least one root shell already running. The main reason why I start my Porteus in multiuser text mode -mode 3 - I have logged into a minimum of one root shell in the Virtual Terminal. And one guest that manually starts X via startx.
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 5795
Joined: 06 Feb 2013, 22:12
Distribution: 4.0 & 5.0rc2 Cinnamon 64 ISOs
Location: Western NY, USA

Ravas coding goodies

Post#20 by Ed_P » 10 Nov 2020, 17:53

Rava wrote:
10 Nov 2020, 07:07
Also, quoted from here http://www.linfo.org/current_directory.html
A single dot is also used to represent the current directory in some commands, scripts (i.e., short programs) and the PATH environmental variable.
Compare guest's $PATH with root's $PATH:

Code: Select all

guest@porteus:/etc/UPower$ echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/games:/opt/porteus-scripts:.
vs

Code: Select all

root@porteus:~# echo $PATH 
/usr/local/sbin:/usr/local/bin:/sbin:/usr/sbin:/bin:/usr/bin
The missing "." for root means it won't execute programs or scripts from the current directory, but in guest's case it will do so.
. scriptname works! :o I've always used ./scriptname. Thank you. :)

Added in 4 minutes 42 seconds:
Rava wrote:
10 Nov 2020, 17:36
^ by reading free bash scripting tutorials / bash scripting books.

______________________________________

from here: [url=viewtopic.php?p=79540#p79540]
That link doesn't point to bash scripting tutorials or books. But it does indicate you didn't learn it in school.
Ed

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#21 by Rava » 10 Nov 2020, 18:02

Ed_P wrote:
10 Nov 2020, 17:57
That link doesn't point to bash scripting tutorials or books. But it does indicate you didn't learn it in school.
I use the divider line _____________ to show that my post switches to a different topic. Just duckduckgo "bash scripting tutorial" :)
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#22 by Rava » 28 Nov 2020, 18:31

Did you know, when you start editing a sh or bash shell script that should use wget to download one file after the other, you can start the file via

Code: Select all

#wget -c "" -O "One.ISO"
#wget -c "" -O "Long.ISO"
#wget -c "" -O "Road.ISO"
and when you found the first URL - let us presume here the ISOs are not all in the same folder, or you have to solve a recaptcha to get the URL for each file, or the site adds some expiration info to the link that makes the URL only valid for a certain time, you can execute the script when you finished all info for the first file:

Code: Select all

wget -c "http://example.com/sillyname.iso" -O "One.ISO"
#wget -c "" -O "Long.ISO"
#wget -c "" -O "Road.ISO"
and while the download is running, you can research the URL for the 2nd ISO and add the next URL as soon as you found it and again save the script:

Code: Select all

wget -c "http://example.com/sillyname.iso" -O "One.ISO"
wget -c "http://example.net/hidden.ISO" -O "Long.ISO"
#wget -c "" -O "Road.ISO"
Cave! Make sure to not delete the first line, bash will read the next line of a script when it did finish the line above, so especially your code has if ; else; elif ; fi or other such constructs, deleting a line above what bash currently is executing or deleting the current executed line will mess up the way bash reads and executes the script.

To be safe, use a function at the beginning if you need such functionality, and below you can add the code bash should run, e.g. lets say you created some mydownload () function that uses two parameters: the URL of the remote file and the file name to store that file locally (and probably does more stuff since all that can be done via wget, no need for a function, this is just as easy example) you can have a script like so:

Code: Select all

function mydownload () {
	wget "$1" -O "$2"
}
mydownload "http://example.com/sillyname.iso" "One.ISO"
#mydownload  "" "Long.ISO"
#mydownload "" "Road.ISO"
And like above, while the script is saved like above and executed, you then can look for the 2nd URL and add it.

Just make sure to never delete a line above what bash currently executes - while the script is executed - you can add as many lines you like, though, but only below the line bash currently executes.

To me it is like a self-build download manager via a very simple script, giving you the ability to start the first download and then you have the time finding the following URLs while the downloads above are done one after the other by themselves. :magic:
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#23 by Rava » 01 Dec 2020, 06:32

One might presume that using a "." in sh or bash or a script as part of a filename automatically makes the "." a regex, e.g. it then stands for any one character.

But it is not so, and here is the proof:

Code: Select all

root@porteus:/backup# mkdir Ed_P_test
root@porteus:/backup# cd Ed_P_test/
root@porteus:/backup/Ed_P_test# type l
l is aliased to `/bin/ls -oa --color=auto --time-style=long-iso'
root@porteus:/backup/Ed_P_test# l
total 28
drwxr-xr-x   2 root  4096 2020-12-01 07:12 .
drwxr-xr-x 649 root 24576 2020-12-01 07:12 ..
root@porteus:/backup/Ed_P_test# touch dummy.exe
root@porteus:/backup/Ed_P_test# touch dummy_exe
root@porteus:/backup/Ed_P_test# l
total 28
drwxr-xr-x   2 root  4096 2020-12-01 07:12 .
drwxr-xr-x 649 root 24576 2020-12-01 07:12 ..
-rw-r--r--   1 root     0 2020-12-01 07:12 dummy.exe
-rw-r--r--   1 root     0 2020-12-01 07:12 dummy_exe
root@porteus:/backup/Ed_P_test# rm *.exe
root@porteus:/backup/Ed_P_test# l
total 28
drwxr-xr-x   2 root  4096 2020-12-01 07:12 .
drwxr-xr-x 649 root 24576 2020-12-01 07:12 ..
-rw-r--r--   1 root     0 2020-12-01 07:12 dummy_exe
So as you can see, if *.exe would translate into [any characters][one character]exe rm *.exe would have deleted both dummy.exe and dummy_exe. But the "." in the filename was understood literally, not regex-ally, and so rm *.exe not deleted dummy_exe.

HTH
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 5795
Joined: 06 Feb 2013, 22:12
Distribution: 4.0 & 5.0rc2 Cinnamon 64 ISOs
Location: Western NY, USA

Ravas coding goodies

Post#24 by Ed_P » 01 Dec 2020, 13:55

Try it with rm *?exe
Ed

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#25 by Rava » 01 Dec 2020, 16:58

Ed_P wrote:
01 Dec 2020, 13:55
Try it with rm *?exe
rm *exe will do the trick.

Also, when addressing hidden files you need to start it with a "."
".*" -> all hidden files
"*" -> all files, but no hidden files.
If you have to address both use ".* *" e.g. cp .* * /target-dir/ copies all hidden files and normal files but would complain about folders in source directory.
And this:

Code: Select all

/bin/cp: -r not specified; omitting directory '.'
/bin/cp: -r not specified; omitting directory '..'
Since copying . or .. would fail anyway, better use .??* instead, but if you had hidden files with only one character after the "." like ".a" .??* would not find it.

Code: Select all

guest@porteus:/tmp/test$ l
total 0
drwxr-xr-x  2 guest 120 2020-12-01 17:49 .
drwxrwxrwt 15 root  460 2020-12-01 17:44 ..
-rw-r--r--  1 guest   0 2020-12-01 17:49 .a
-rw-r--r--  1 guest   0 2020-12-01 17:49 .b
-rw-r--r--  1 guest   0 2020-12-01 17:43 .hidden
-rw-r--r--  1 guest   0 2020-12-01 17:43 normal
guest@porteus:/tmp/test$ l .??*
-rw-r--r-- 1 guest 0 2020-12-01 17:43 .hidden
guest@porteus:/tmp/test$ l .?* -d
drwxrwxrwt 15 root  460 2020-12-01 17:44 ..
-rw-r--r--  1 guest   0 2020-12-01 17:49 .a
-rw-r--r--  1 guest   0 2020-12-01 17:49 .b
-rw-r--r--  1 guest   0 2020-12-01 17:43 .hidden
guest@porteus:/tmp/test$ 
l is aliased to the same as in my last example above.
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 5795
Joined: 06 Feb 2013, 22:12
Distribution: 4.0 & 5.0rc2 Cinnamon 64 ISOs
Location: Western NY, USA

Ravas coding goodies

Post#26 by Ed_P » 01 Dec 2020, 17:32

Rava wrote:
01 Dec 2020, 16:58
Ed_P wrote:
01 Dec 2020, 13:55
Try it with rm *?exe
rm *exe will do the trick.
Not the point.

ls .* * has interesting results.
Ed

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#27 by Rava » 01 Dec 2020, 19:16

Ed_P wrote:
01 Dec 2020, 17:32
Rava wrote:
01 Dec 2020, 16:58
Ed_P wrote:
01 Dec 2020, 13:55
Try it with rm *?exe
rm *exe will do the trick.
Not the point.
Since when? Look at the thread title, it is the point, unless we are in e.g. "Ed_Ps coding goodies". Here I decide what's the point, but feel free creating "Ed_Ps coding goodies" or whatever you wanna call it, then you take the shots and you give directions.
Ed_P wrote:
01 Dec 2020, 17:32
ls .* * has interesting results.
you refer to the reason I used

Code: Select all

ls .* * -d 
instead of a mere

Code: Select all

ls .* 
above?

(I used l .?* -d but the reason for ls -d is the same, also

Code: Select all

alias l='/bin/ls -oa --color=auto --time-style=long-iso'
)
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 5795
Joined: 06 Feb 2013, 22:12
Distribution: 4.0 & 5.0rc2 Cinnamon 64 ISOs
Location: Western NY, USA

Ravas coding goodies

Post#28 by Ed_P » 01 Dec 2020, 20:49

a. You were talking about the unique regex "." and how it is used to delete files. I brought up the "?" regex and how it can be used for deleting files. You then said that the simple *exe expression could be use to delete all exe files. "*" is common and well known so referring to it was imo not the point of unique "." and "?" regexs.

b. I was also impressed with your ls .* * -d command and when I tried it I forgot the -d and was amazed at what it produced on my system.

c. Forums, including this one, are for discussions where people exchange ideas and help others solve problems.

d. This is not your forum, you post something, you will receive responses that you may, or may not, agree with. And if you post something that a mod doesn't agree with your posting, or thread, may be deleted. Standard forum protocol.

:)
Ed

User avatar
Rava
Contributor
Contributor
Posts: 2676
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 5.0rc1 x86-64 XFCe
Location: Forests of Germany

Ravas coding goodies

Post#29 by Rava » 02 Dec 2020, 02:51

In this explanation I will not explain what the special folder names of "." and ".." stand for, I have already explained that elsewhere. For short: "." is an alias for the current directory, ".." is one for the next folder below in the folder hierarchy. E.g. when you are in a folder named /tmp/test then "." would be a reference to /tmp/test itself, and ".." would be a reference to "/tmp"
______________________________________________________
Ed_P wrote:
01 Dec 2020, 20:49
a. You were talking about the unique regex "." and how it is used to delete files.
No, I brought now two examples explaining that in usual every day bash or sh use the "." is read literally. Not as regex, or in other words: the exact opposite of what you presumed above.

My first example illustrating the literal use of "." not the regex one:
When using *.exe to address filenames for bash and sh that translate into
"*" "any character, also none character, but not the new line character[¹]"
"." "a literal '.' - not 'one character' "
"exe" "exe"

Let us presume the following alias for l and the following test files:

Code: Select all

root@porteus:/tmp/test# alias l
alias l='/bin/ls -oa --color=auto --time-style=long-iso'
root@porteus:/tmp/test# l
total 0
drwxr-xr-x  2 guest 160 2020-12-02 03:08 .
drwxrwxrwt 16 root  480 2020-12-01 19:58 ..
-rw-r--r--  1 root    0 2020-12-02 03:08 .dummy_exe
-rw-r--r--  1 root    0 2020-12-02 03:00 .exe
-rw-r--r--  1 root    0 2020-12-02 03:00 d.exe
-rw-r--r--  1 root    0 2020-12-02 03:00 dummy.exe
-rw-r--r--  1 root    0 2020-12-02 03:01 dummy_exe
-rw-r--r--  1 root    0 2020-12-02 03:00 exe
Now, let us test my hypothesis:

Code: Select all

root@porteus:/tmp/test# l *.exe
-rw-r--r-- 1 root 0 2020-12-02 03:00 d.exe
-rw-r--r-- 1 root 0 2020-12-02 03:00 dummy.exe
root@porteus:/tmp/test# l .*exe
-rw-r--r-- 1 root 0 2020-12-02 03:08 .dummy_exe
-rw-r--r-- 1 root 0 2020-12-02 03:00 .exe
root@porteus:/tmp/test# l .exe
-rw-r--r-- 1 root 0 2020-12-02 03:00 .exe
root@porteus:/tmp/test# l .* -d
drwxr-xr-x  2 guest 160 2020-12-02 03:08 .
drwxrwxrwt 16 root  480 2020-12-01 19:58 ..
-rw-r--r--  1 root    0 2020-12-02 03:08 .dummy_exe
-rw-r--r--  1 root    0 2020-12-02 03:00 .exe
root@porteus:/tmp/test# l .?* -d
drwxrwxrwt 16 root 480 2020-12-01 19:58 ..
-rw-r--r--  1 root   0 2020-12-02 03:08 .dummy_exe
-rw-r--r--  1 root   0 2020-12-02 03:00 .exe
root@porteus:/tmp/test# l .??* -d
-rw-r--r-- 1 root 0 2020-12-02 03:08 .dummy_exe
-rw-r--r-- 1 root 0 2020-12-02 03:00 .exe
I hope you now finally see what I mean.

I recommend all users to only use hidden files or folders that have at least two characters after the initial "." that marks the file (or folder) as hidden.
By adhering to that simple rule, you can exclude the special folder names of "." and ".." by using ".??*" to address your file and folders.
".?*" would always also find the special folder of ".." - as seen in the above example.

And by the above example we also covered the area of my previous post second example: the one addressing hidden or dot-files: meaning any files or or folders - but also the special folders "." and ".." - that start with a "." - which by UNIX and Linux specifications is what marks these as hidden.
Of course you are free to create hidden files and folders that contain two characters that start with a "." but are not the special folder name of "." like ".a" or ".1" or ".;", but like I wrote above, that would hinder you in using a mere ".??*" to address these files and folders while excluding the special folder ".."

Of course there are ways in using e.g. ".a" as file name or folder name and still address all such files or folders while excluding the special folder ".." by using regex, but that is not what this very post and my last two posts are about, so I omit it for now.

Again, with addressing the hidden file issue, bash and sh read a file or folder name starting with a "." not as "one character", but as a literal ".", always.
______________________________________________________
Ed_P wrote:
01 Dec 2020, 20:49
b. I was also impressed with your ls .* * -d command and when I tried it I forgot the -d and was amazed at what it produced on my system.
Because of the special folders ".." and "."
______________________________________________________
Ed_P wrote:
01 Dec 2020, 20:49
c. Forums, including this one, are for discussions where people exchange ideas and help others solve problems.

d. This is not your forum, you post something, you will receive responses that you may, or may not, agree with.
I think you misunderstood of what I meant by what I wrote, and you not even quoted what you are referring to.

All I meant is that the thread is about me giving coding tricks and advice, so when I define a topic like the literal reading of "." not a regex one then what I gave as examples is to illustrate the point. If you want to discuss a different approach showing the regex use of "." you are free to do so, even in here.
But on the other hand, no one is forced to answer to anything she or he does not want to. That is what I meant: I started this thread, and while others can discuss anything in here that is on-topic, meaning Ravas coding, not just any coding (see how the thread is named very specifically, not named as a generic "let us discuss any coding topics" subject), they can do so. But I not have to answer when I do not want to, that was meant by me writing you should start your own thread.

And if someone would start discussing any coding stuff that derails the threads subject - "Ravas coding goodies", I could delete such posts because they would be off-topic here. Again: be aware of the very special way the subject is named.
The correct thread for such would be a generic subject like "coding tips" or whatnot, and not "Ravas coding goodies".

I hope we finally cleared up all misunderstandings!


__________________
[¹] One or more newline characters can be a legitimate part of any UNIX / Linux file or folder name, but even more so than just a whitespace, newline characters break most scripts because the coder never anticipated that a file or folder name can be composited of more than one line.
Anyone who coded one-liners for sh or bash, or coded scripts for sh or bash encountered the issues with file or folder names containing whitespace(s), but that issue is easy to handle, compared to the can of worms if you want to make your script fit handling file or folder names containing newline character(s) or in other words: more than one line of text info as name.
Cheers!
Yours Rava

Post Reply