Self-updating PorteuX

[rych]

porteux, my script tries to update all existing system xzms like this:

Code: Select all

URL=https://api.github.com/repos/porteux/porteux/releases/latest
CH=current #channel
DIR=$BASEDIR/porteux #installation directory

for x in "05-devel-$CH" "06-crippled_sources" "08-multilanguage-$CH" "08-nvidia" "0050-multilib-lite-$CH"; do
    for d in "modules" "optional"; do
        if compgen -G "$DIR/$d/$x*" > /dev/null; then
            echo "updating $x in $d"
            mkdir -p $TMP/$d #backup
            mv $DIR/$d/$x* $TMP/$d/
            url=$(curl -s $URL | grep $x | grep url | cut -f 4 -d '"')
            wget -N -nv --show-progress $url -P $DIR/$d
        fi
    done
done

File names are matched as $x* locally or as grep $x when extracting its url -- because you offer all xzms already extracted from the corresponding zips, all except 08-nvidia -- it's still inside nvidia-driver zip.

It's no big problem, I'll just treat it as a special case. I was just curious why you offered each of them in both xzm and zip form, all except the poor, second class citizen 08-nvidia

[rych]

Also, I guess I have to identify (equate) channel name to the kernel version when retrieving the 2 modules:

06-crippled_sources-6.13.1.xzm
08-nvidia-570.86.16-k.6.13.1-porteux-x86_64.xzm

because unlike other modules that explicitly include channel name, e.g. 08-multilanguage-current-20250202.xzm, those 2 instead have the kernel version 6.13.1. My script has to know that for example 6.13.1 is the same as current. If I were lazy and spoiled, I'd ask you to rename tthem to follow the same pattern

06-crippled_sources-current-6.13.1.xzm
08-nvidia-current-570.86.16-k.6.13.1-porteux-x86_64.xzm

[porteux]

Maybe you're over-complicating things. Using the script I sent you before, you can add this:

Code: Select all

wget https://github.com/porteux/porteux/releases/download/${porteuxLatestVersion}/crippled-sources.zip
wget https://github.com/porteux/porteux/releases/download/${porteuxLatestVersion}/nvidia-driver-current.zip

[Ed_P]

Using the script I sent you before,

Personally I love that script. Useful, helpful, convenient. I did add 2 minor tweaks to it though; 1. before the "if" as I suggested previously
echo "Current=$porteuxCurrentVersion"
echo "Latest=$porteuxLatestVersion"
and 2. at the end "read", so I can run the script from the file manager.

To support users of the nvidia-driver adding an if to check if their current system has it then include it in the update downloads should make the script universal.

Added in 16 minutes 51 seconds:

Code: Select all

#!/bin/bash

# https://forum.porteus.org/viewtopic.php?p=102067#p102067 - porteux
repository=porteux
project=porteux
porteuxChannel=current
desktopEnvironment=lxde

porteuxCurrentVersion=$(cat /etc/os-release | grep VERSION | cut -f 2 -d '"' | head -1)
porteuxLatestVersion=$(curl -s https://github.com/${repository}/${project}/tags/ | grep "/${repository}/${project}/releases/tag/" | grep -oP "(?<=/${repository}/${project}/releases/tag/)[^\"]+" | uniq | grep -v "alpha" | grep -v "beta" | grep -v "rc[0-9]" | grep -v "master." | grep -v "clutter-keyring-dialog" | head -1)
echo "Current=$porteuxCurrentVersion"
echo "Latest=$porteuxLatestVersion"
if [[ ${porteuxLatestVersion//v} > ${porteuxCurrentVersion} ]]; then
	porteuxIso=$(curl -s https://api.github.com/repos/${repository}/${project}/releases/latest | grep ${porteuxChannel} | grep ${desktopEnvironment} | grep url | cut -f 4 -d '"')
	wget ${porteuxIso}
    if [ -e /mnt/live/memory/images/*nvidia* ]; then
       wget https://github.com/porteux/porteux/releases/download/${porteuxLatestVersion}/crippled-sources.zip
       wget https://github.com/porteux/porteux/releases/download/${porteuxLatestVersion}/nvidia-driver-current.zip 
    fi
fi
read
exit

[rych]

Maybe you're over-complicating things

Ah yes, the kernel version is the the same for the both stable and current channels, sorry I didn't realize. So that's no issue.

Using the script I sent you before

Yes, I'm building on top of your script, thank you.

To support users of the nvidia-driver adding an if to check if their current system has it then include it in the update downloads

Yes, my script updates whichever additional system xzms it finds in modules/ or optional/, nvidia is zip only so need to unzip myself, others are currently offered as xzm too so I use those straight.

Another curiosity: porteux, the nvidia-driver-current.zip and nvidia-driver-stable.zip seem to contain exactly the same 08-nvidia-570.86.16-k.6.13.1-porteux-x86_64.xzm 428,468 KB? Is that always the case? If nvidia driver is only (?) kernel dependent, and since kernel version is and always be (?) the same in both stable and current, why keep two identical assets? thanks

[porteux]

Another curiosity: porteux, the nvidia-driver-current.zip and nvidia-driver-stable.zip seem to contain exactly the same 08-nvidia-570.86.16-k.6.13.1-porteux-x86_64.xzm 428,468 KB? Is that always the case? If nvidia driver is only (?) kernel dependent, and since kernel version is and always be (?) the same in both stable and current, why keep two identical assets? thanks

They're identical at the moment, but we decided to have separated files in case we need to create different versions of the driver for stable and current.

[rych]

I now have this script that updates PorteuX installation running under porteus: I don't really need to update porteux in-place -- I can always re-boot into porteus for that.

Code: Select all

#!/bin/bash
# Offline PorteuX system updater
# Runs under porteus
# Assumes PorteuX in /porteux
# and in /mnt/sda1/boot/porteux

URL=https://api.github.com/repos/porteux/porteux/releases/latest
CH=current #channel
DE=lxde #desktop environment
DIR=$BASEDIR/porteux #installation directory
BOOT=${BASEDIR%?}1/boot/porteux #boot files location: initrd.zst, vmlinuz

#zip encapsulation is the server preference
declare -A module #local-remote (nick)name mapping
module[05-devel]=devel-$CH.zip
module[06-crippled_sources]=crippled-sources.zip
module[08-multilanguage]=08-multilanguage-$CH #exists as only xzm
module[0050-multilib]=multilib-$CH.zip
module[08-nvidia]=nvidia-driver-$CH.zip

#Finding out current local and latest remote version numbers
TMP=$BASEDIR/porteux #for now
mkdir -p $TMP/001-core
mount $DIR/base/001-core*.xzm $TMP/001-core -t squashfs -o loop
current=$(cat $TMP/001-core/etc/os-release | grep VERSION_ID | cut -f 2 -d '=')
umount $TMP/001-core
rm -d $TMP/001-core
#tail removes the 'v' from 'v1.9'
latest=$(curl -s $URL | grep -oP '"tag_name": "\K.*?(?=")' | tail -c+2)
echo "$latest > $current"

if [[ $latest > $current ]]; then
    echo
    TMP=$BASEDIR/porteux/backup$current
    mkdir -p $TMP/download
    echo "Getting latest ISO"
    url=$(curl -s $URL | grep $CH | grep $DE | grep url | cut -f 4 -d '"')
    wget -N -q --show-progress $url -P $TMP/download
    ISO=$TMP/download/$(basename "$url" .iso)
    mkdir -p $ISO
    mount $ISO.iso -o ro $ISO
    echo "Updating base"
    rm -rf $TMP/base
    mv $DIR/base $TMP/
    cp -r $ISO/porteux/base $DIR/
    mv $BOOT/{initrd.zst,vmlinuz} $TMP/
    cp $ISO/boot/syslinux/{initrd.zst,vmlinuz} $BOOT/
    echo "Updating other modules"
    for m in "${!module[@]}"; do
        for d in "modules" "optional"; do
            if compgen -G "$DIR/$d/$m*" > /dev/null; then
                echo "updating $m in $d"
                mkdir -p $TMP/$d #backup
                mv $DIR/$d/$m* $TMP/$d/
                m="${module[$m]}" #remote name selector
                url=$(curl -s $URL | grep $m | grep url | cut -f 4 -d '"')
                wget -N -q --show-progress $url -P $TMP/download
                m=$(basename "$url") #downloaded filename
                if [[ "${m##*.}" == "zip" ]]; then
                    unzip $TMP/download/$m -d $DIR/$d
                else
                    cp $TMP/download/$m $DIR/$d/
                fi
            fi
        done
    done
    umount $ISO
    rmdir $ISO

    echo "$(tput setaf 2) " #green and hopefully a new line
    echo "Old files are saved in"
    ls $TMP -RA1

    echo "$(tput bold) " #bold and a new line
    read -t 10 -n 1 -r -p "Would you like to delete the entire $TMP? [y/N] " response
    if [[ $response == [yY] ]]; then
        rm -r $TMP
    fi
    echo "$(tput sgr0)"
fi

https://t.me/PorteusX/98/100

[rych]

Code: Select all

root@porteus:~# update_porteux 
2.0 > 1.9

Getting latest ISO
porteux-v2.0-curren 100%[===================>] 486.00M  12.6MB/s    in 43s     
Updating base
Updating other modules
updating 05-devel in optional
devel-current.zip   100%[===================>] 171.64M  11.5MB/s    in 13s     
Archive:  /mnt/sda5/porteux/backup1.9/download/devel-current.zip
 extracting: /mnt/sda5/porteux/optional/05-devel-current-20250401.xzm  
updating 0050-multilib in optional
multilib-current.zi 100%[===================>] 126.54M  9.98MB/s    in 11s     
Archive:  /mnt/sda5/porteux/backup1.9/download/multilib-current.zip
 extracting: /mnt/sda5/porteux/optional/0050-multilib-lite-current-20250401.xzm  
updating 08-nvidia in optional
nvidia-driver-curre 100%[===================>] 420.69M  9.41MB/s    in 37s     
Archive:  /mnt/sda5/porteux/backup1.9/download/nvidia-driver-current.zip
 extracting: /mnt/sda5/porteux/optional/08-nvidia-570.133.07-k.6.14.0-porteux-x86_64.xzm  
 
Old files are saved in
/mnt/sda5/porteux/backup1.9:
base
download
initrd.zst
optional
vmlinuz

/mnt/sda5/porteux/backup1.9/base:
000-kernel-6.13.1.xzm
001-core-current-20250202.xzm
002-gui-current-20250202.xzm
002-xtra-current-20250202.xzm
003-lxde-0.10.1-current-20250202.xzm

/mnt/sda5/porteux/backup1.9/download:
devel-current.zip
multilib-current.zip
nvidia-driver-current.zip
porteux-v2.0-current-lxde-0.10.1-x86_64.iso

/mnt/sda5/porteux/backup1.9/optional:
0050-multilib-lite-current-20250202.xzm
05-devel-current-20250202.xzm
08-nvidia-570.86.16-k.6.13.1-porteux-x86_64.xzm
 
Would you like to delete the entire /mnt/sda5/porteux/backup1.9? [y/N] 
root@porteus:~# update_porteux 
2.0 > 2.0
root@porteus:~# 

[porteux]

So it worked flawlessly?

[rych]

porteux, Yes my update_porteux seems to work, thanks for not changing the paths or names on the server. And this is an off-line updater as you can see: I run it from porteuS.

Perhaps unrelated problem is that now it first loads into Guest graphical mode even though I have login=root. So I have to Logout/Switch users (or Ctrl-Alt-F6 e.g. login as root there and issue startx) -- then my familiar root environment is loaded as before. But all the modules seem to have updated successfully. And I only take your new initrd.zst, vmlinuz and use them with my own grub -- maybe there are other files I need from the new iso/boot ?

Anyway, a question to you -- who's loading me into Guest desktop I assume it's by default, but I do specify login=root /etc/bootcmd.cfg confirms it.

[porteux]

That's definitely a bug you found.

Please, edit /etc/rc.d/rc.4 and replace this

Code: Select all

/etc/lxdm/lxdm.conf

With this:

Code: Select all

/etc/lightdm/lightdm.conf

It should fix the issue.

Sorry about that. It's going to be officially fixed in the next release.

Thanks for reporting

[rych]

It should fix the issue

Yes... Thanks... It logs in into root Desktop now. But there is still something special about it: for example, it doesn't pick up my /root/.Xresources setting of Xft.dpi: 120, which is immediately noticeable as UI is small. I still have to do same as before

Ctrl-Alt-F6 e.g. login as root there and issue startx)

in order to get it right and big again.

By the way, (now?), it turns out I don't even need to explicitly set login=root cheatcode: it logs in as root by default anyway. Both of those config files must have autologin=root or something. So, I've removed login=root and still get root Desktop.

But anyway, the Xft.dpi: 120 isn't respected on the initial Desktop. That file, /root/.Xresources, is a part of my custom module porteux/optional/DPI/1920x1080.xzm which is loaded via load=1920x1080 cheatcode.

[porteux]

@rych, thanks for keeping reporting bugs! You're clear and gentle, and that's very helpful!

To fix the .Xresources bug, edit your /etc/lightdm/Xsession and add the following just before the last line (dbus-run-session...):

Code: Select all

userresources=$HOME/.Xresources
usermodmap=$HOME/.Xmodmap
sysresources=/etc/X11/xinit/.Xresources
sysmodmap=/etc/X11/xinit/.Xmodmap

if [ -f $sysresources ]; then
    xrdb -merge $sysresources
fi

if [ -f $sysmodmap ]; then
    xmodmap $sysmodmap
fi

if [ -f $userresources ]; then
    xrdb -merge $userresources
fi

if [ -f $usermodmap ]; then
    xmodmap $usermodmap
fi

Now, regarding your sessions being logged in as root even when you don't use the login cheatcode, I guess you're saving the session automatically. If you take a look at /etc/rc.d/rc.4 it actually changes a file (using sed), so if you're saving the session it will include this change, therefore the next time you boot it will get this file change and will log in as root. I see this is not what you expect, but to be honest I'm not sure if there's a way to log in as root by just passing parameters to LightDM or any other solution that doesn't involve changing a system file. If you have any ideas, that would be great!

[rych]

To fix the .Xresources bug, edit your /etc/lightdm/Xsession and add the following

Thanks porteux, but I've decided to stop patching the system files for now. Because 1) you're going to release a new update soon anyway -- and I will forget to delete all the custom patches from porteux/changes.

2) the topic being "Self-updating PorteuX", the acumulated /changes reused from previous versions may lead to conflicts -- at least /changes/etc and other system folders should be fresh (we don't top-up new chicken to the pan with the left-over chicken from last week) with every new release -- basically, I wish I only needed to save /changes/root for my user config etc.

For example, if the system wants to write autologin-user=root into /changes/etc/lightdm or /changes/etc/lxdm I'm going to omit those files in changes-exit.conf -- which ironically is itself preserved in /changes/etc/changes-exit.conf -- so there we go, unavoidably, I have to keep some modified system directories inside my changes but I want to keep it to a minimum at least.

So, I leave the patching of system files to you if you please So that when I update_porteux I don't have to remember which patches to undo from the /changes

When I update Porteus once every 2 years I actually start with empty /changes and slowly move bits and pieces from /changesOLD. Since I don't want to do that with every frequent PorteuX update, and instead simply reuse the same /changes, I'd better keep them minimal and careful.

By the way, for your next release, with PorteuX always aiming at speed, would you experiment perhaps with LZ4 compression as discussed here: dir2lz4 anyone? for your base modules? The jury is still out on LZ4 vs. Zstd for SquashFS, so tests and testimonies are needed. Thanks again for your work,

[porteux]

@rych, I see what you mean. But instead of patching the system using rootcopy or savechanges, you can just extract the module that contains the file that needs to be changed, make the change then recreate the module. This way the next time a new PorteuX version is released you don't need to worry.

Interesting this LZ4. I conducted some tests and I'm not sure if it's good option at the moment. What I did was combining all base modules + devel into a single folder and then I measured the time to decompress the whole thing (since compression times don't matter when using the system):

zstd PorteuX default:
0m1.070s
647.1 MB

lz4 default:
0m0.891s
1.0 GB

lz4 high compression default block size:
0m0.886s
881.8 MB

lz4 high compression 256 block:
0m0.891s
866.1 MB

We're talking about LZ4 being ~200ms faster to extract the whole system + devel. That's virtually no difference. However, the increase in size is significant.

In a more realistic scenario where the system just decompresses the files that are needed by demand, let's say during boot time, we would be talking about less than 10 ms faster than zstd, which is literally impossible to notice. I know I'm using a fast CPU. But imagine a CPU that is 10 times slower: we would be talking about 100 ms difference, which is still nothing.

What you think?