Updating/building modules in Nemesis

Arch based Porteus community project
Post Reply
User avatar
M. Eerie
Shogun
Shogun
Posts: 243
Joined: 31 Aug 2017, 21:18
Distribution: APorteus BUDGIE x64

Updating/building modules in Nemesis

Post#1 by M. Eerie » 08 Aug 2019, 16:34

From the works of Jack and ncmprhnsbl, I've been sketching some lines to build/update modules on Nemesis. Just putting it all togheter here to save new users from digging around.

Please, feel free to improve it and report back, as I am not a scripting master at all.

Code: Select all

#!/bin/bash
##
## Update module from $PORTDIR/modules by searchinng part of his name
##

## Check for root
if [ `whoami` != "root" ]; then
      echo "You must be admin to run this script"
      exit
fi

read -p "Input part of module name to update: " modinput
modulo=`basename $(ls $PORTDIR/modules | grep -i $modinput) .xzm`

PATH1=/tmp/$modulo
unsquashfs -f -d $PATH1 $PORTDIR/modules/$modulo.xzm

## 1. Preparation and ensure the cache is clean
# update pacman database
pacman -Sy

# clear package cache
rm /var/cache/pacman/pkg/*

#needed to update packages in an arbitrary path
cp --parents -au /var/lib/pacman/local/ALPM_DB_VERSION $PATH1

OUT1=/tmp/lx
OUT2=/tmp/upall
OUT3=/tmp/uplx
pacman -Qqr $PATH1 | awk '{print $1}' > $OUT1
pacman -Qqu | awk '{print $1}' > $OUT2
sleep 5

## Compare module to find upgrades   
comm -12 $OUT2 $OUT1 > $OUT3

## 2. Download the packages to cache without dependency checking 
## pacman -Sw / Swdd <list of packages> 
## (dd) --> disable dependency checking
pacman -Sddw `cat $PWD/uplx`

## 3. Install the Updated packages to the extracted module
## (dd) --> disable all dependency checking
pacman -Uddr $PATH1 /var/cache/pacman/pkg/*.pkg.tar.xz

## 4. Stripping unneeded files
#......................
# lxstrip.sh BEGIN
#......................

## Some of this stuff could be useful to someone
#cp -au --parents $PATH1/usr/include $PATH2/usr/
#cp -au --parents $PATH1/usr/share/info $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/doc $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/gtk-doc $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/licenses $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/man $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/locale $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/gir-1.0 $PATH2/usr/share/

rm -rf $PATH1/usr/include/* 
rm -rf $PATH1/usr/share/info/* 
rm -rf $PATH1/usr/share/doc/* 
rm -rf $PATH1/usr/share/gtk-doc/* 
rm -rf $PATH1/usr/share/licenses/* 
rm -rf $PATH1/usr/share/man/* 
rm -rf $PATH1/usr/lib/systemd/*
rm -rf $PATH1/tmp
#rm -rf $PATH1/usr/share/gir-1.0/*
#rm $PATH1/usr/bin/gtk3-demo
#rm $PATH1/usr/bin/gtk3-demo-application
#rm $PATH1/usr/bin/gtk3-widget-factory

#rm $PATH1/usr/share/applications/gtk3-demo.desktop
#rm $PATH1/usr/share/applications/gtk3-widget-factory.desktop


rm $PATH1/var/lib/pacman/local/ALPM_DB_VERSION
[[ -f $PATH1/var/log/pacman.log ]] && rm $PATH1/var/log/pacman.log

#......................
# lxstrip.sh END
#......................

## 5. Make the module
[[ -d $PATH1/home ]] && chown guest.users $PATH1/home
chown -R root.root $PATH1
dir2xzm $PATH1/ $PATH1.xzm

## 6.Clean the cache before updating any other modules 
pacman -Scc
rm $OUT1 $OUT2 $OUT3
rm -r $PATH1
unset modinput modulo OUT1 OUT2 OUT3 PATH1
:hmmm: Maybe lacks a cd /tmp before unpacking...

Updated. Still dirty work..., but you get the idea. :)

Thanks.
Last edited by M. Eerie on 09 Aug 2019, 19:48, edited 2 times in total.

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 2690
Joined: 20 Mar 2012, 03:42
Distribution: 5.0rc1-64bit all-DE+more
Location: australia
Contact:

Updating/building modules in Nemesis

Post#2 by ncmprhnsbl » 09 Aug 2019, 05:32

some suggests:
M. Eerie wrote:
08 Aug 2019, 16:34
Maybe lacks a cd /tmp before unpacking...
or use /tmp instead of $PWD

pacman -Scc requires user input[yes/no], possibly just rm /var/cache/pacman/pkg/*

one problem that can occur with automating this way is if there is some new dependency appears, it will be missed..
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

User avatar
M. Eerie
Shogun
Shogun
Posts: 243
Joined: 31 Aug 2017, 21:18
Distribution: APorteus BUDGIE x64

Updating/building modules in Nemesis

Post#3 by M. Eerie » 09 Aug 2019, 19:58

ncmprhnsbl wrote:
09 Aug 2019, 05:32
if there is some new dependency appears, it will be missed..
We force disabling dependencies (pacman -Sdd). I don't know how to deal with this. Is there any workaround?

By now, If the module suddenly fails, it's up to the user to try pacman -S <package> and see what are the new dependencies needed.

:oops:

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 2690
Joined: 20 Mar 2012, 03:42
Distribution: 5.0rc1-64bit all-DE+more
Location: australia
Contact:

Updating/building modules in Nemesis

Post#4 by ncmprhnsbl » 09 Aug 2019, 21:56

really depends on the packages involved, what is trying to be achieved..
for example, when i update the 003-xfce module, there are some gtk3 deps(eg. adwaita-icon-theme..etc) that i don't want/need..
another approach could be to use --ignore <packages> variable
this kind of script was really only meant to streamline upgrading of a desktop type module...to save rebuilding from scratch each time..
for most software, just using pman should be sufficent..
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

User avatar
M. Eerie
Shogun
Shogun
Posts: 243
Joined: 31 Aug 2017, 21:18
Distribution: APorteus BUDGIE x64

Updating/building modules in Nemesis

Post#5 by M. Eerie » 20 May 2020, 08:48

New approach...

This one is intended for base modules only.

I'd appreciate some feedback. Please, let me know if it works for you.

LAST UPDATED: Jun 3, 2020

Code: Select all

#!/bin/bash
# Update NEMESIS base modules ( $PORTDIR/base )
# <https://forum.porteus.org> 

function sqtype() { unsquashfs -s $@ | awk 'NR==4 {printf " " $2 " "}; NR==5 {print}'; }; export -f sqtype
#for mod in $MODDIR/*; do printf "$(basename $mod)\t"; sqtype $mod; done
function lssq() { unsquashfs -ll "$1" | sed 's/squashfs-root//g'; }; export -f lssq

HERE="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
warn() { echo; echo "$(tput setaf 1) $1 $(tput sgr0)"; echo; }
confirm() { echo; tput setaf 6; read -n1 -rsp "$1"; tput sgr0; echo -e "\n"; [[ $REPLY = [Yy] ]]; }

[ ! $(grep -o 'base' /proc/cmdline) ] && echo "Please init Nemesis with **Always Fresh** boot paramaters" && exit 0;
[ ! $EUID != 0 ] || sudo $(basename $0)

the_end() {
	if [ $? == 0 ]; then
		dir2xzm /tmp/LOCALES /tmp/LOCALES.xzm
		mv /tmp/'*.xzm' $HERE
		cd /tmp; rm -r LOCALES $outMOD >/dev/null 2>&1
		echo -ne "  All Done! \n Your files are here: $HERE"
	else 
		warn " Aborted. Exit code: $?"
	fi
	unset patches HERE outMOD
	eval "exit $?"
}
trap the_end SIGHUP SIGINT SIGTERM

fixPERMS() {
	chmod -f 750 $1/root #$1/etc/sudoers.d/ $1/etc/openvpn/client/ $1/etc/openvpn/server/
	chmod -f 755 $1/etc $1/etc/local.d $1/usr $1/usr/local $1/usr/local/bin
	find $1/usr/share/icons -type d -exec chmod 755 {} + >/dev/null 2>&1
	find $1/usr/share/icons -type f -exec chmod 644 {} + >/dev/null 2>&1
}

prepareSYS() {
fixPERMS ""

[ $(pacman -Qu | wc -l) == 0 ] || {
	setup-pman # WORKAROUND In case setup-pman fails --> "Please ensure you have a connection and try again. Exit code: 1"
	if [ $? == 1 ]; then
		pacman-db-upgrade && pacman -Sy
		haveged; sleep 3; pacman-key --init; pacman-key --populate archlinux artix; pkill haveged; update-ca-trust;
	fi
	pacman -Syyu --overwrite '*'
	touch /tmp/dbDONE ### avoid iterations
	}
	echo "  System updated!"; echo
}

updateBASE() {
	[ -f /tmp/dbDONE ] || prepareSYS
	[ ! -d /tmp/LOCALES ] && mkdir -p /tmp/LOCALES
	PS3=$'\n'$'\e[00;36mPlease, select base module to update or exit (1-5): \e[0m'
	select outMOD in core gui xtra xfce exit
	do
		patches=""
		case $outMOD in
			core) patches="{archlinux,artix}-{keyring,mirrorlist} {lib,}p11-kit gnupg file open{sysusers,tmpfiles} dbus{,-openrc} ca-certificates";;
			 gui) ;;
			xtra) patches="srt vmaf";; #TODO Can be used to interact with the user and add new deps.
			xfce) ;;
			exit*) the_end;;
		esac
	outMOD="$(basename $(ls -1 $PORTDIR/base | grep -i $outMOD) | cut -d- -f-2)"
	break
	done

cd /tmp
	unsquashfs -f -d $outMOD $PORTDIR/base/$outMOD*
	fixPERMS "$outMOD"

### PATCH CORE MODULE. TRICKY ONE AND NOT 100% WORKING AS post transactions hooks (eudev, openrc...) need to run in real root environment, (mountpoints, etc.)
	if [ $outMOD == "001-core" ]; then
		pacman -Rddr $outMOD {archlinux,artix}-{keyring,mirrorlist}
		rm -r $outMOD/etc/pacman.d/gnupg
		pacman-key --gpgdir $outMOD/etc/pacman.d/gnupg --init
		pacman-key --gpgdir $outMOD/etc/pacman.d/gnupg --populate archlinux artix
	fi

	cp -au --parents /var/lib/pacman/{sync/,local/ALPM_DB_VERSION} $outMOD
	pacman -Qqnr $outMOD | pacman --root $outMOD -Sdd - --overwrite '*'
	# UNDO previous patch / apply new deps
	[ -z "$patches" ] || eval "pacman -Sddr $outMOD $patches"

cd $outMOD
	find usr/share/locale \( -name 'e[ns]_[EU]S' -o -name 'e[ns]' -o -iname 'locale.alias' \) -exec cp -au --parents {} /tmp/LOCALES \;
	rm -r usr/share/{doc,gir-1.0,gtk-doc,info,licenses,locale,man} usr/include usr/lib/{pkgconfig,systemd} tmp boot >/dev/null 2>&1
	rm -r var/lib/pacman/{sync/,local/ALPM_DB_VERSION} >/dev/null 2>&1
	rm etc/ld.so.cache etc/sudo_logsrvd.conf etc/shadow.pacnew etc/sudo.conf

cd /tmp
	mv $outMOD/var/log/pacman.log $HERE/$outMOD-pacman.log >/dev/null 2>&1
	chown -R 0.0 $outMOD; [ -d $outMOD/home ] && chown -R 1000.1000 $outMOD/home >/dev/null 2>&1
	dir2xzm $outMOD "$outMOD"-$(date '+%Y%m%d').xzm
	rm -r $outMOD/
}

while confirm "Shall we update [Yy/*]? "; do updateBASE; done

the_end
NOTES:
  • Code refined. Pacman is such an amazing tool.
  • Some tasks involving creation of users accounts (sudo users, etc.) were skipped according to logs.
  • Additional note: Suggested updating method is from bottom to top. That is: xfce, then xtra, then gui, then core.
By now I've succeeded to get into Desktop with various methods.
When I have more time, I will do some tests and try to bring a final version...
I think that will be all for now.

If I'm not mistaken, the only pending thing is sudo account. I'll be back. B)

Cheers!
Last edited by M. Eerie on 03 Jun 2020, 21:54, edited 30 times in total.

User avatar
M. Eerie
Shogun
Shogun
Posts: 243
Joined: 31 Aug 2017, 21:18
Distribution: APorteus BUDGIE x64

Updating/building modules in Nemesis

Post#6 by M. Eerie » 28 May 2020, 10:25

Testing needed.

Anyone for the task?

Cheers!

User avatar
M. Eerie
Shogun
Shogun
Posts: 243
Joined: 31 Aug 2017, 21:18
Distribution: APorteus BUDGIE x64

Updating/building modules in Nemesis

Post#7 by M. Eerie » 03 Jun 2020, 21:41

Testing still needed.

:roll:

Post Reply