Save change to modules

[benjibasson]

Hi,
I would like to save change only in xzm modules

So I launch the script /opt/porteus-scripts/save-changes
It only save session without shadow pssword, neither /etc/NetworkManager/system-connections/ neither rc.d

=> I change

Code: Select all

#dir2xzm $ch $1
mksquashfs $ch $1 -e dev tmp mnt

by

Code: Select all

zstdFlags="-comp zstd -b 256K -Xcompression-level 22"
mksquashfs $ch $1 -e dev mnt sys tmp var $zstdFlags

doesn't add any setting in my 999-change.xzm althought it is working in porteuX live usb


OK, I use this way
-Save current Porteus Session ( same resulte as porteus's save-change script) in 999-change.xzm
-Save some setting in a module enable every cases even bluetooth in 999-setting.xzm

it save .etc/shadow but doesn't save rc.d neither /etc/NetworkManager/system-connections/

is there a solution to select more setting to save? can I add something in

Code: Select all

mksquashfs $ch $1 -e dev mnt sys tmp var

?

Cheers

[Ed_P]

In Porteus when I save settings I use a script to copy the files and folders I want to save to a /mod folder I create on the /tmp drive then use dir2xzm to create the module.

Here's some of the script for files I save:

Code: Select all

if [ -d /tmp/mod/ ]; then
   rm -rf /tmp/mod
fi
mkdir -p /tmp/mod

# copy modified files

#                  -- Network files --
#touch                 /var/lib/dhcpcd/dhcpd.leases
cp -a -p --parents    /etc/NetworkManager/system-connections/*       /tmp/mod/
cp -a -p --parents    /var/run/NetworkManager/*                      /tmp/mod/
cp -a -p --parents    /var/lib/NetworkManager/*                      /tmp/mod/
cp -a -p --parents    /etc/resolv.conf                               /tmp/mod/
cp -a -p --parents    /etc/NetworkManager/*                          /tmp/mod/
cp -a -p --parents    /var/lib/xkb                                   /tmp/mod/

And this is how I create the module:

Code: Select all

dir2xzm  /tmp/mod  /tmp/$MOD.xzm   &&   rm -rf /tmp/mod

echo 
ls -sh   /tmp/*.xzm

[Rava]

^
What Ed_P did not state in his post but should be obvious: The module on /tmp/$MOD.xzm needs to be copied somewhere prior reboot or it will be lost.
E.g. like so:

Code: Select all

cp -v /tmp/$MOD.xzm $PORTDIR/base/

This works when your $PORTDIR/base/ is writeable. When that is inside an ISO and you boot via an ISO it will not work. (I never booted via ISO - I use the extracted xzm files into local folder(s) approach so I cannot test what $PORTDIR looks like when you boot from an ISO but others who boot via ISO can tell you)

[Vic]

I like to make a module manually whenever I can. Create a folder and save the folders and files I want in it and in KDE right click dir2xzm. At least that way I have some control and the new module only has what I want.

If I need more I create more modules and then merge, or I create a new folder with all the items and make a new module.

Vic

[roro]

@ED_P

I'm still a beginner/newbie
I also want to understand the script.

mod is the folder with the many files.
But what is $MOD?
MOD with capital letters?
Thanks

[Ed_P]

MOD is a variable I use to assign a name to the module I'm creating higher up in my script.

Code: Select all

MOD=mychanges   # mychanges-$(date +%Y%m%d)

What Ed_P did not state in his post but should be obvious: The module on /tmp/$MOD.xzm needs to be copied somewhere prior reboot or it will be lost.

Yes. Thank you for pointing that out. If USB booting put it in the /porteus/modules/ folder or the /porteus/optional/ folder. If ISO booting put it in a folder that you reference with the extramod= cheatcode when booting.

I like to make a module manually whenever I can.

Me too.

[Rava]

Yes. Thank you for pointing that out.

That part might be obvious to you and me, but for obvious reasons not for newbies to scripting. (And one can be a Linux user for years and still be a noob when it comes to scripting)

I like to make a module manually whenever I can. Create a folder and save the folders and files I want in it and in KDE right click dir2xzm. At least that way I have some control and the new module only has what I want.

In my book, that's the best approach. Could be that in the beginning you miss some files and only realize that after reboot that most settings and stuff are present while some others are missing.

E.g. these files

Code: Select all

/home/guest/.config/mimeapps.list
/home/guest/.config/dconf/user

could be overlooked.

What I like is the approach with using is syntax

Code: Select all

 cp -av --parents 

(my is just slightly more compact)
What does it do? -v tells you what is done
--parents also creates the parents folder if they did not exist previously. Important since else all files would just sit in /tmp/mod/ - instead of e.g. mimeapps.list being in /tmp/mod/home/guest/.config/mimeapps.list
And files all being in /tmp/mod/ would not function the way you want.

I'm still a beginner/newbie
I also want to understand the script.

I hope we are able to teach you good.

[Vic]

@roro, I am also a beginner/newbie.

Hopefully this is not off topic resulting in death. Opposite of the problem in this post, usually the save settings script works great. I save settings to a module and upon reboot if successful, use xzm2dir to see what it did. Then I split the folder(s) / file(s) up and make different test modules.

Reboot one test module at a time.

Eventually I find the exact folder(s) / file(s) I want and then make a new module. The save settings script always includes way more than I want , so that is my way of shrinking it down. Tedious but fun. Always have fun.

Vic

[rych]

I would like to save change only in xzm modules

I save changes for backup purpose with this short script:

Code: Select all

ch=$PORTDIR/changes
chxzm=/tmp/changes.xzm
COMP_ZSTD="-comp zstd -b 256K"
mksquashfs $ch $chxzm $COMP_ZSTD -noappend -e dev tmp mnt root/.cache

No need to copy stuff around or call other scripts.

I've checked it now and mine does contain the files and directory (not sure what is "shadow pssword") you're missing, so that strange. Try not copying first, but reading straight off the original path?

[Rava]

Eventually I find the exact folder(s) / file(s) I want and then make a new module. The save settings script always includes way more than I want , so that is my way of shrinking it down. Tedious but fun. Always have fun.

I run a different approach.

I use a script that can be started as either make-991-usr_local_bin.sh or make-992-rootcopy.sh

Code: Select all

root@rava:/usr/local/bin# ls -o make-99*
-rwxr--r-- 1 root 2939 2022-11-08 21:30 make-991-usr_local_bin.sh
lrwxrwxrwx 1 root   25 2021-02-27 23:06 make-992-rootcopy.sh -> make-991-usr_local_bin.sh

The path where it finds its source directory is hardcoded into the script.

I have a helper script that gives me some examples for copying new or changed files to the correct locations (as in the hardcoded paths described above)

after I copy new file(s) I either run make-991-usr_local_bin.sh or make-992-rootcopy.sh .

It copies the resulting module (with added YYYY-MM-DD part of its name) into a certain folder and there is a symlink that gets changed to the currently created module name.
In my $PORTDIR/base are also two symlinks that point to the symlinks in the above folder, and that symlink again points to the most recent files, so I can use a multitude of porteus/base folders and all will use the most up-to-date settings-modules automatically.

Why do I split up my settings into two files?

Because 991-usr_local_bin_YYYY-MM-DD.xzm are meant to be used universally (holding scripts and some programs and also very few global settings to be used in all DEs) while 992-rootcopy_5.0_YYYY-MM-DD.xzm are meant for one specific DE only. in my case that is XFCE, when creating a 992-rootcopy* module for a different DE I would put that DE into its name, but since my standard is XFCE (and I know it) I refrain from adding that info to the 992-rootcopy* module.

Code: Select all

root@rava:/Porteus_modules# ls -o 99*2023*
-rw-r--r-- 1 root 167936 2023-04-05 09:45 991-usr_local_bin_2023-04-05.xzm
-rw-r--r-- 1 root 167936 2023-09-07 12:06 991-usr_local_bin_2023-09-07.xzm
-rw-r--r-- 1 root 167936 2023-09-08 13:40 991-usr_local_bin_2023-09-08.xzm
-rw-r--r-- 1 root 380928 2023-08-31 11:29 992-rootcopy_5.0_2023-08-31.xzm
-rw-r--r-- 1 root 380928 2023-09-03 09:39 992-rootcopy_5.0_2023-09-03.xzm
-rw-r--r-- 1 root 380928 2023-09-08 09:16 992-rootcopy_5.0_2023-09-08.xzm

Might sound complicated but it is very handy and easy to use once you hardcode your needed paths reflecting your personal partitions (you need an ext[234] partition to hold all the files or else when using VFAT or Ntfs your file permissions would be messed up)

I thought of sharing the whole project but for now got told that no one would be interested in it.

But when you are interested let me know and I post a demo-version of the scripts (that would not work out of the box since I cannot know which paths you want to use and that info is to be hardcoded into the script once at the beginning, then you never need to update the script when you keep yourc chosen paths the same)

[Ed_P]

Finally I can signon and post without a form error.

That part might be obvious to you and me

Oh I've forgotten to do it once or twice.

(And one can be a Linux user for years and still be a noob when it comes to scripting)

Yeah I know.

This is a complete script I use to save my nemo bookmarks that I like to my new NVMe drive since I boot ISOs which are non-writable.

Code: Select all

#!/bin/bash

# Save changes as a mod file 

# https://forum.porteus.org/viewtopic.php?f=81&t=5981&start=15#p46643

MOD=mynemobkmks   # mychanges-$(date +%Y%m%d)
TO=Optional
SYSTM=
ISOBOOT=
#set -x;
# http://forum.porteus.org/viewtopic.php?f=53&t=3801&start=30#p28472
BOOT=`grep -A1 "Porteus data found in:" /var/log/porteus-livedbg|tail -n1|sed 's^//^/^g'`
DRV=${BOOT:5:4}

if [ "${BOOT:0:12}" == "/mnt/isoloop" ]; then
   ISOBOOT=`grep -A1 "//porteus" /var/log/porteus-livedbg`
   SYSTM="${ISOBOOT:11:10}/"
   DRV=${ISOBOOT:5:4}
fi
if [ "$DRV" == "nvme" ]; then
   SYSTM="${ISOBOOT:16:10}/"
   DRV=${ISOBOOT:5:9}
fi 

CHANGES=/mnt/$DRV/$SYSTM$TO/$MOD
if [ "$1" == "r" ] && [ "$2" != "" ] ; then
   CHANGES="$2"
fi

if [ `whoami` != "root" ]; then
   echo toor | sudo -S  echo "******"
   sudo sh $0 $1 $2; exit
fi

if [ -d /tmp/mod/ ]; then
   rm -rf /tmp/mod
fi
mkdir -p /tmp/mod

# copy modified files

cp -a --parents    /home/guest/.config/dconf/*                    /tmp/mod/
cp -a --parents    /etc/resolv.conf                               /tmp/mod/

#                  -- File Manager - nemo - bookmarks --
cp -a --parents          /root/.config/nemo*                      /tmp/mod/
cp -a --parents          /root/.config/gtk-3.0/*                  /tmp/mod/
cp -a --parents    /home/guest/.config/nemo/*                     /tmp/mod/
cp -a --parents    /home/guest/.config/gtk-3.0/*                  /tmp/mod/

#                  -- File Manager - xed behavior, power settings & clock display --
cp -a --parents    /home/guest/.config/dconf/user                 /tmp/mod/

du -h    /tmp/mod/   | grep "[0-9]M"

echo
echo Changed files copied. Press Enter to create module, Ctrl+C to exit.
echo

if [ -f $CHANGES*.xzm ]; then
   CF=`ls -lght --time-style long-iso $CHANGES*.xzm | awk '{print $5,$6}'`
   if [ ${CF:11:2} -lt 12 ]; then
      CF=$CF"am"
   else
      PM=$((${CF:11:2} - 12))
      if [ $PM == 0 ]; then PM=12; fi
      CF=${CF:0:11}$PM${CF:13}"pm"
   fi
   CFS=`ls -lght --time-style long-iso $CHANGES.xzm | awk '{print $4}'`
   echo "Current file:" $CFS $CF
   read
fi

dir2xzm  /tmp/mod /tmp/$MOD.xzm   &&   rm -rf /tmp/mod

echo 
ls -sh   /tmp/*.xzm

DIR () {
ls -lght --time-style long-iso $1 | awk '{print $5,$6,$4,$7}'
}

echo
echo Press Enter to copy /tmp/$MOD.xzm to $SYSTEM$TO, Ctrl+C to exit.
DIR     /tmp/$MOD.xzm 
if [ -f $CHANGES*.xzm ]; then
   DIR  $CHANGES*.xzm
fi 
read
if [ -f $CHANGES*.xzm ]; then
   mv   $CHANGES*.xzm  $CHANGES.xyzm
fi
cp      /tmp/$MOD.xzm  /mnt/$DRV/$SYSTM$TO
sleep 1
rm      /tmp/$MOD.xzm
echo Finished!
sleep 1

usually the save settings script works great.

I save changes based on finding them using some commands I've found here: viewtopic.php?f=81&t=1612&p=11168#p11168, here: viewtopic.php?f=39&t=132 and here: viewtopic.php?p=86214#p86214. Once I know what's changed I make a module of those files.
This is my changes.sh script.

Code: Select all

#!/bin/sh
# https://forum.porteus.org/viewtopic.php?f=81&t=3776#p27204
  
PSWD=toor    # password for root access

X=1          # default search time in minutes
if [ ! "$1" == "" ]; then
   X=$1
fi  

echo " mmin = "$X

echo $PSWD | sudo -S  echo "******" 
if [ "$2" == "" ]; then
   sudo -S find /mnt/live/memory/changes  -iname "*" -mmin -$X
else
#  https://forum.porteus.org/viewtopic.php?p=86214#p86214 
   sudo -S find /mnt/live/memory/changes  -not -path "*/.cache/*" -iname "*" -mmin -$X  
fi
read
exit

Added in 2 hours 5 seconds:

Code: Select all

root@rava:/Porteus_modules# ls -o 99*2023*
-rw-r--r-- 1 root 167936 2023-04-05 09:45 991-usr_local_bin_2023-04-05.xzm
-rw-r--r-- 1 root 167936 2023-09-07 12:06 991-usr_local_bin_2023-09-07.xzm
-rw-r--r-- 1 root 167936 2023-09-08 13:40 991-usr_local_bin_2023-09-08.xzm
-rw-r--r-- 1 root 380928 2023-08-31 11:29 992-rootcopy_5.0_2023-08-31.xzm
-rw-r--r-- 1 root 380928 2023-09-03 09:39 992-rootcopy_5.0_2023-09-03.xzm
-rw-r--r-- 1 root 380928 2023-09-08 09:16 992-rootcopy_5.0_2023-09-08.xzm

Doesn't loading so many versions of the same files make your booting process take longer? When I create a new mychanges.xzm module I rename the old one mychanges.xyzm to prevent it from loading but still being available to return to if the new one has problems.

[Rava]

Doesn't loading so many versions of the same files make your booting process take longer? When I create a new mychanges.xzm module I rename the old one mychanges.xyzm to prevent it from loading but still being available to return to if the new one has problems.

Like I wrote:

It copies the resulting module (with added YYYY-MM-DD part of its name) into a certain folder and there is a symlink that gets changed to the currently created module name.
In my $PORTDIR/base are also two symlinks that point to the symlinks in the above folder, and that symlink again points to the most recent files, so I can use a multitude of porteus/base folders and all will use the most up-to-date settings-modules automatically.

The script that creates the most recent version also creates the symlink pointing to the most recent version.

I like having older versions available in the slim chance that one of these modules contains some weird error.

It is like so:
${PORTDIR}/base/991-usr_local_bin_RECENT.xzm -> /mnt/sda99/Porteus_modules/5.0/991-usr_local_bin_RECENT.xzm
and in /mnt/sda99/Porteus_modules/5.0/:
991-usr_local_bin_RECENT.xzm -> 991-usr_local_bin_2023-09-08.xzm

And thus all base folders where the 991-usr_local_bin_RECENT.xzm links to the link of /mnt/sda99/Porteus_modules/5.0/991-usr_local_bin_RECENT.xzm links always to the most recently created version - in my case at the moment, to 991-usr_local_bin_2023-09-08.xzm

Thus by having my settings-module-creating scripts create not only the most recent module, but also the always named identical symlink pointing to the most recent module, the modules can keep their YYYY-MM-DD info in their names, still, at every boot only the most recent version is loaded (as it should be, since it could happen you want settings files removed for some reason and not just updated; in that case loading older settings modules would mess that up).

The main trick is using a symlink to a symlink to a file.

The main part is I never have to bother updating the boot media / boot device /base/ folder since that symlink can stay the same and still always points to the most recently created module.

Added in 7 minutes 9 seconds:
And all that works flawlessly for years for me. As you can see, the first version got created around "2021-02-27 23:06" (time stamp of the symlink) and the most recent version of make-991-usr_local_bin.sh is 2022-11-08 21:30 - so unchanged in approx 10 months.

Because when you set it all up, you only have to use the cp command copying the file (with all its base folders) to the correct destination that got hardcoded into the script, then run the script (after you are satisfied that you updated or copied all new files you want included) and the most recent module got created, plus the symlink to that module.

Added in 8 minutes 54 seconds:
Of course when you want a boot media that holds all modules on itself so that it can be used at any PC, you still would have to manually update the settings-modules.

Scripting can do much, and symlink to symlink to file can do a great deal of (semi-automatic) work for you, but it cannot copy files to some random external partition it not knows about.

But if you need that functionality, you can always expand my make-991-usr_local_bin.sh script.

[Ed_P]

So basically you don't save your modules in the /porteus/modules folder and don't use the extramod= cheatcode. An interesting/complicated approach.

[Rava]

An interesting/complicated approach.

It might sound complicated at the very beginning, but once it is set up it means it can be automated and works by itself.

[Rava]

This is how easy it works as soon as it is set up:

create a new script using a template

Code: Select all

root@rava:/# cp /usr/local/bin/help.EMPTY /usr/local/bin/help.mpg321
root@rava:/# l3afpad /usr/local/bin/help.mpg321

Testing the new script:

Code: Select all

guest@rava:/sound$ help.mpg321
help.mpg321 V0.1
create playlist via make-m3u-playlist.sh FOLDERNAME
edit FOLDERNAME.m3u if needed.

Silent output:
mpg123 -q@ FOLDERNAME.m3u

Normal output:
mpg123 -@ FOLDERNAME.m3u
guest@rava:/sound$ 

Copy the new script to the correct location as set up in my make-991-usr_local_bin.sh / make-992-rootcopy.sh script:

Code: Select all

root@rava:/# cp -av --parents /usr/local/bin/help.mpg321  /mnt/sda5/Porteus_modules/5.0/rootcopy+Ulb/991-usr_local_bin_RECENT/
'/usr/local/bin/help.mpg321' -> '/mnt/sda5/Porteus_modules/5.0/rootcopy+Ulb/991-usr_local_bin_RECENT/usr/local/bin/help.mpg321'

I have a helper-script named help.cp that gives me not only the two target folders (one for make-991-usr_local_bin.sh and one for make-992-rootcopy.sh ) but also the names of some often used settings files

and just run the script:

Code: Select all

root@rava:/# make-991-usr_local_bin.sh
make-991-usr_local_bin.sh V0.4
loading usr_local_bin settings…
$MODPATH= /mnt/sda5/Porteus_modules/5.0
$MODBASEFLD= rootcopy+Ulb/991-usr_local_bin_RECENT
$MYMODULE= 991-usr_local_bin_2023-09-19.xzm
$MYMODLNKRECENT= 991-usr_local_bin_RECENT.xzm
$MYSEARCH= 991-usr_local_bin_*
_____________________________________________________________________

$PWD is now: /mnt/sda5/Porteus_modules/5.0 …
/opt/porteus-scripts/dir2xzm rootcopy+Ulb/991-usr_local_bin_RECENT 991-usr_local_bin_2023-09-19.xzm
Parallel mksquashfs: Using 8 processors
Creating 4.0 filesystem on 991-usr_local_bin_2023-09-19.xzm, block size 262144.
[===============================================================-] 358/358 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
	compressed data, compressed metadata, compressed fragments,
	compressed xattrs, compressed ids
	duplicates are removed
Filesystem size 161.84 Kbytes (0.16 Mbytes)
	19.09% of uncompressed filesystem size (847.96 Kbytes)
Inode table size 4852 bytes (4.74 Kbytes)
	35.59% of uncompressed inode table size (13634 bytes)
Directory table size 4412 bytes (4.31 Kbytes)
	50.91% of uncompressed directory table size (8667 bytes)
Number of duplicate files found 2
Number of inodes 400
Number of files 358
Number of fragments 4
Number of symbolic links 28
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 14
Number of ids (unique uids + gids) 3
Number of uids 2
	root (0)
	guest (1000)
Number of gids 2
	root (0)
	users (100)
/bin/ln -sfv 991-usr_local_bin_2023-09-19.xzm 991-usr_local_bin_RECENT.xzm
'991-usr_local_bin_RECENT.xzm' -> '991-usr_local_bin_2023-09-19.xzm'
167936 2023-09-14 21:41 991-usr_local_bin_2023-09-14.xzm
167936 2023-09-15 06:52 991-usr_local_bin_2023-09-15.xzm
167936 2023-09-18 11:11 991-usr_local_bin_2023-09-18.xzm
167936 2023-09-19 04:49 991-usr_local_bin_2023-09-19.xzm
    32 2023-09-19 04:49 991-usr_local_bin_RECENT.xzm -> 991-usr_local_bin_2023-09-19.xzm
make-991-usr_local_bin.sh V0.4 - all seems well. Exiting now.
root@rava:/# 

The real output of my script and of the helper-script help.cp is coloured to make the output more easy to read.

So in the end, while the coding of my script-suite took some time and thinking, the setup itself is easy and quickly done to suit one's own system's need. But most importantly when it is set up the day-to-day effort needed and time it takes to run it while creating the most recent version of my settings module is minimal and takes mere seconds.
And the last part was my goal.

Added in 12 minutes 8 seconds:
Or in other words: Mission accomplished:

Code: Select all

yt-dlp lJ7F2kWLGuE