[HOWTO] create a safe script changing system critical file

Post tutorials, HOWTO's and other useful resources here.
User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

[HOWTO] create a safe script changing system critical file

Post#1 by Rava » 23 Sep 2015, 21:23

Hello fellow sailors, and welcome aboard the SS Porteus for this fun and (maybe?) bumpy ride.

This howto is meant to show all or most of the theoretically ideas when creating a system relevant script that needs to be as secure and bug free as possible.

It is also some kind of a way to show a non-programmer or any person who so far did not create a more complex script by her/himself by the way of a documented HOWTO including the programmer's thoughts on the matter, and especially the details and file or program or system components the programmer has to consider when creating a script, especially one that changes such a crucial file as /etc/fstab.



The issue that I already posted somewhere on here in hope of someone of the core developers being able to have a solution to the changing of /etc/fstab bug after resume/thaw it this:

In my Porteus 3.1 XFCe version, when I suspend (hibernate won't work on that machine that is known having a ... weird BIOS, e.g. even with the newest available BIOS update, I sure can install 4GB of RAM in the motherboard, but only 3 1/2 GB are supported. Some RAM checking software [like the one included with Knoppix, and me thinks older versions of Slax and Porteus had that feature of RAM check instead of booting some OS, but for some reason that was removed. [I would like to tweak all my Porteus to have that RAM checking thingy again, but not figured out what exactly that is, I tried but ran into some issues that gave me headache, that I no longer can recall; tl;dr: I was not able to include this RAM checker in recent Porteus, what I recall is that there are 2 kinds of software that share the same name, or 2 different versions of the same software, the older one that has the RAM check, and the other/newer one that changed the software, so that the RAM check is removed and the software part now does something different, but something plausible due to it's name being not a clear indicator that this thingy does a RAM check at boot time... </digress>]

So, that RAM checker of old(er) told me that indeed 4 GB are installed, but even that RAM tester could only test 3 1/2 GB of RAM.

Code: Select all

# grep /dev/zram0 /etc/fstab
is either

Code: Select all

/dev/zram0 none swap sw,pri=1 0 0
or

Code: Select all

/dev/zram0 none swap sw,pri=1000 0 0
When it's the last variant, then all is okay and nothing needs to be fixed.
When it's the first variant, some [so far unknown] software that was run at either resume/thaw or the suspend/hibernate stage (most probably during wakeup aka resume/thaw, me presumes) did this change. (The "unknown" software could just be the standard updating /etc/fstab component of Porteus / XFCe Porteus that fails to recall that when booted fresh, the priority of /dev/zram0 is way higher than the standard 1, and is so out of good reason, cause RAM based swap > harddisk based swap.)

When I had to kill X because of system freeze via Alt, SysRq + RE (aka REISUP, see also System request), some background programs are also killed and when starting X anew the system is not that same as the original XFCe, e.g. you have to start dhcpd manually, or the NetworkManager icon displaying the current network state is missing, also Hibernate or Suspend via the GUI menu is no longer available, I have to run as root pm-supend manually.
When I had to kill X cause of the freeze (could be the bug in the well known NVidia 6???
The current machine has a GeForce 6200 LE card, but on another machine I have another GeForce 6??? card that gives the same error after a while, running X and Porteus or days. The freeze error can be circumvented by exiting X and restarting X. When the card has no graphic data and reverts to the text based virtual console, then its almost full graphics card RAM is emptied by itself, and I can again use Porteus for some days or week(s) until the next freeze shows its ugly buggy head. :bad:

This 6??? class card driver that causes the RAM of the card to be filled up without removing old unneeded data, at at some time the system freezes cause the graphics card is unable to work due to that bug that sadly never was debugged by Nvidia, as far as I know all cards of that "class" suffer from the very same bug, not due to the card, aka the hardware being faulty but just because of the software driver bug, *facepalm*



Aaaaaanyhow, the need to kill X so brutally sure gives hints of what might cause /etc/fstab to be changed after each boot. Since /etc/fstab is no longer updated after a needed Alt, SysRq + RE and restart of X.



Now, my theory so far for some missing components is that some stuff was not started, cause some scripts in the /etc/rc.d directory are non-executable by default, still at normal boot they are executed by the system, but are not executed cause of missing non-executable attribute after the Alt, SysRq + RE hack...
[That is my theory of why some components malfunction or are missing or need to be run manually]

All I recall of malfunctioning parts are these:
* NetworkManager not running, and it cannot be run manually, seems it missed another component that needs to be started already prior invoking NetworkManager.
* dhcpd needs to be run manually, that is only needed after the first resume/thaw, then dhcpd is the background demon and handles going online or connecting/disconnecting the wired network when I unplug or plug-in the network cable manually...
* Hibernate and Suspend missing from the XFCe menu, but still pm-hibernate itself working like it should.
* Some other parts of XFCe settings manager is either missing or malfunctioning. Since currently I run a non-REISUP'd Porteus I cannot check out what the missing / malfunctioning XFCe settings manager component is; Still I am quite sure I recall it being the PowerManager (or parts of it) that is either missing (aka when clicked, nothing happens, but the icon is still there in XFCe settings manager) or it could be started but some of the most crucial parts are either greyed out or plain missing when run. [Last I tried that is now many months ago, and I just start my dhcpd and so the pm-suspend manually and know that I miss the NetworkManager icon, and have to look at the network cable itself for checking if the missing internet is due to a router or switch or Internet Provider issue, or due to just unplugged network cable.]

...


So, back on topic. When I had to kill X and restarted it, the software part that handles updating /etc/fstab is missing. E.g. I have to create the mount point folder manually and also have to mount a device manually when I plug in some storage, e.g. an external harddisk or an USM stick ("USB Flashdisk").

For now, after manually editing /etc/fstab and changing the /dev/zram0 from 1 to 1000 (the boot default was 100, not 1000, but that's not an issue at all, both are valid parameters, and in this case, both do the same, that is, prioritizing the RAM based swap and putting the harddisk based swap at the tail end of the priority chain.

Here is, as end for now, the info I get after I edited the needed change into /etc/fstab manually:

Code: Select all

root@porteus:/usr/local/bin# free -m|grep Swap;swapon -a;swapon -s;fx
Swap:            0          0          0
Filename				Type		Size	Used	Priority
/dev/sda1                               partition	2097148	0	1
/dev/zram0                              partition	694968	0	1000
23.09.2015 22:44:36 ____________________________________________________________
             total       used       free     shared    buffers     cached
Mem:          3393       3246        146          0         11        221
-/+ buffers/cache:       3013        379
Swap:         2726          0       2726
Please recall that I created an aliases an a function; both I use on a daily basis. One is fx, the other dx:

Code: Select all

root@porteus:/usr/local/bin# type fx
fx is aliased to `echo $(date +%d.%m.%Y\ %H:%M:%S) ____________________________________________________________;free -m'
root@porteus:/usr/local/bin# type dx
dx is a function
dx () 
{ 
    echo $(date +%d.%m.%Y\ %H:%M:%S) ____________________________________________________________;
    df -T -Tm $* | grep --color=auto -vE 'devtmpfs|/mnt/live/run'
}
I had to change dx from a mere alias into a function, cause I wanted it to omit the unneeded (and unwanted) lines about devtmpfs and /mnt/live/run.


For now, that's all folks.


Now that we have all the background of why an automated tweak /etc/fstab is needed out of the way, we finally can approach the next stage: That, my dear fellow seamen and sailors would be starting to finally ponder about how to implement the edit / diff /etc/fstab via script in a as failsafe manner as we can make it.


If you have any ideas on that, on how you would solve the automated /etc/fstab diff / edit matter, please post a reply. Please do reply especially if you see some error or mistake in this post so far... <B
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Re: [HOWTO] create a safe script changing system critical fi

Post#2 by Rava » 25 Sep 2015, 07:52

Now that we have all the background explanations finally behind us, on to the next stage.


What do you suggest would be the best approach? Check if in /etc/fstab zram0 is set to pri=1 and if it's set to pri=1, then change it?

Or is another approach a better one, especially since we want the script to be as secure as possible, and only tinker with a system critical file (etc/fstab here) when absolutely necessary.
Cheers!
Yours Rava

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Re: [HOWTO] create a safe script changing system critical fi

Post#3 by Rava » 12 Oct 2015, 00:50

^
Of course, the described solution is a stupid one.

It doesn't matter at all if zram0 is set to pri=1 or to pri=100; what matters at the beginning is: is zram and the hard disk swap activated (check with swapon -s) or not.

If the swap that should be active is set, then we don't need to do anything, especially not change such a system critical file like /etc/fstab.

_______________________________________

Due to my personal RL issues, I currently have not the time to really create the automated safe script changing system critical file that I planned when first creating this thread.

My quick and dirty solution is this script. I leave out the test if the user who started the script by setting the owner to root.root and the permission to only be readable and executable by the owner:

Code: Select all

# ls -o /usr/local/bin/swapx 
-rwx------ 1 root 415 Oct  8 01:44 /usr/local/bin/swapx
Here the quick and dirty solution, that needs manual edit of /etc/fstab, but at least the rest is automated:

Code: Select all

#!/bin/sh
VERSION="0.1"
MYNAME="swapx"
# set me to: chown root.root	chmod 0700/-rwx------
test ! "$ECHO_COLORS"x = "x" && test -f $ECHO_COLORS && . $ECHO_COLORS
echo -e "${bld}${yel}$MYNAME$off V$yel$VERSION${off} ❤❤❤ mcedit /etc/fstab;swapon -a;swapon -s;fx ❤❤❤"
mcedit /etc/fstab
swapon -a
swapon -s
echo $(date +%d.%m.%Y\ %H:%M:%S) ____________________________________________________________
free -m
The colour management I use for most of my scripts I described already elsewhere. In short, ECHO_COLORS is a systemwide parameter that holds the name to the file that defines the colours in a human readable form (thus making coding scripts more easy to debug, and also makes the scripts better human readable)

The current file looks like so:

Code: Select all

#!/bin/bash
# Maintained since 1998. 
# Cave! You should no longer refer to it as /usr/local/bin/echo.colors! Use the
# system variable $ECHO_COLORS instead which is the FQPN to this file.
# details see echo.farben if installed; or else: man ls; man console_codes
bld="\033[1m"	# bold  	[0;1m <- as true escape-sequence working even WITHOUT echo -e
und="\033[4m"	# underline	[0;4m
fla="\033[5m"        # flashing 	[0;5m
red="\033[31m"	# red		[0;31m
gre="\033[32m"	# green   	[0;32m
yel="\033[33m"	# yellow  	[0;33m
blu="\033[34m"	# blue		[0;34m
mag="\033[35m"	# magenta	[0;35m
cya="\033[36m"	# cyan		[0;36m
#spc="\033[60G"	# flush-right  (column 60)
off="\033[m"	# off [0;m
Please be aware that in current Porteus man console_codes is no longer installed, but the alternative, looking up missing man pages via Lynx works like a charm. As you can see, this concept of colouring scripts dates back to 1998, which predates Porteus by mere 12 years, and has seen many Linux variants, including Slax and GParted Live CD. When I recall right, it was GParted that forced me to change my hardcoded filename of /usr/local/bin/echo.colors to the parameter $ECHO_COLORS because that OS not supported a writeable/usable /usr/local/bin, which is not how a system should behave, since the /local/bin paths are meant to be used for local additions, like the name suggests... (Could be another live CD that I used for a short time, quite some years back, that did have this strange behaviour, and not GParted, but I am not really sure which one it was, and it's not really important anyway...)
Image
Cheers!
Yours Rava

Post Reply