Page 1 of 1

Updating/building modules in Nemesis

Posted: 08 Aug 2019, 16:34
by M. Eerie
From the works of Jack and ncmprhnsbl, I've been sketching some lines to build/update modules on Nemesis. Just putting it all togheter here to save new users from digging around.

Please, feel free to improve it and report back, as I am not a scripting master at all.

Code: Select all

#!/bin/bash
##
## Update module from $PORTDIR/modules by searchinng part of his name
##

## Check for root
if [ `whoami` != "root" ]; then
      echo "You must be admin to run this script"
      exit
fi

read -p "Input part of module name to update: " modinput
modulo=`basename $(ls $PORTDIR/modules | grep -i $modinput) .xzm`

PATH1=/tmp/$modulo
unsquashfs -f -d $PATH1 $PORTDIR/modules/$modulo.xzm

## 1. Preparation and ensure the cache is clean
# update pacman database
pacman -Sy

# clear package cache
rm /var/cache/pacman/pkg/*

#needed to update packages in an arbitrary path
cp --parents -au /var/lib/pacman/local/ALPM_DB_VERSION $PATH1

OUT1=/tmp/lx
OUT2=/tmp/upall
OUT3=/tmp/uplx
pacman -Qqr $PATH1 | awk '{print $1}' > $OUT1
pacman -Qqu | awk '{print $1}' > $OUT2
sleep 5

## Compare module to find upgrades   
comm -12 $OUT2 $OUT1 > $OUT3

## 2. Download the packages to cache without dependency checking 
## pacman -Sw / Swdd <list of packages> 
## (dd) --> disable dependency checking
pacman -Sddw `cat $PWD/uplx`

## 3. Install the Updated packages to the extracted module
## (dd) --> disable all dependency checking
pacman -Uddr $PATH1 /var/cache/pacman/pkg/*.pkg.tar.xz

## 4. Stripping unneeded files
#......................
# lxstrip.sh BEGIN
#......................

## Some of this stuff could be useful to someone
#cp -au --parents $PATH1/usr/include $PATH2/usr/
#cp -au --parents $PATH1/usr/share/info $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/doc $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/gtk-doc $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/licenses $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/man $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/locale $PATH2/usr/share/
#cp -au --parents $PATH1/usr/share/gir-1.0 $PATH2/usr/share/

rm -rf $PATH1/usr/include/* 
rm -rf $PATH1/usr/share/info/* 
rm -rf $PATH1/usr/share/doc/* 
rm -rf $PATH1/usr/share/gtk-doc/* 
rm -rf $PATH1/usr/share/licenses/* 
rm -rf $PATH1/usr/share/man/* 
rm -rf $PATH1/usr/lib/systemd/*
rm -rf $PATH1/tmp
#rm -rf $PATH1/usr/share/gir-1.0/*
#rm $PATH1/usr/bin/gtk3-demo
#rm $PATH1/usr/bin/gtk3-demo-application
#rm $PATH1/usr/bin/gtk3-widget-factory

#rm $PATH1/usr/share/applications/gtk3-demo.desktop
#rm $PATH1/usr/share/applications/gtk3-widget-factory.desktop


rm $PATH1/var/lib/pacman/local/ALPM_DB_VERSION
[[ -f $PATH1/var/log/pacman.log ]] && rm $PATH1/var/log/pacman.log

#......................
# lxstrip.sh END
#......................

## 5. Make the module
[[ -d $PATH1/home ]] && chown guest.users $PATH1/home
chown -R root.root $PATH1
dir2xzm $PATH1/ $PATH1.xzm

## 6.Clean the cache before updating any other modules 
pacman -Scc
rm $OUT1 $OUT2 $OUT3
rm -r $PATH1
unset modinput modulo OUT1 OUT2 OUT3 PATH1
:hmmm: Maybe lacks a cd /tmp before unpacking...

Updated. Still dirty work..., but you get the idea. :)

Thanks.

Updating/building modules in Nemesis

Posted: 09 Aug 2019, 05:32
by ncmprhnsbl
some suggests:
M. Eerie wrote:
08 Aug 2019, 16:34
Maybe lacks a cd /tmp before unpacking...
or use /tmp instead of $PWD

pacman -Scc requires user input[yes/no], possibly just rm /var/cache/pacman/pkg/*

one problem that can occur with automating this way is if there is some new dependency appears, it will be missed..

Updating/building modules in Nemesis

Posted: 09 Aug 2019, 19:58
by M. Eerie
ncmprhnsbl wrote:
09 Aug 2019, 05:32
if there is some new dependency appears, it will be missed..
We force disabling dependencies (pacman -Sdd). I don't know how to deal with this. Is there any workaround?

By now, If the module suddenly fails, it's up to the user to try pacman -S <package> and see what are the new dependencies needed.

:oops:

Updating/building modules in Nemesis

Posted: 09 Aug 2019, 21:56
by ncmprhnsbl
really depends on the packages involved, what is trying to be achieved..
for example, when i update the 003-xfce module, there are some gtk3 deps(eg. adwaita-icon-theme..etc) that i don't want/need..
another approach could be to use --ignore <packages> variable
this kind of script was really only meant to streamline upgrading of a desktop type module...to save rebuilding from scratch each time..
for most software, just using pman should be sufficent..

Updating/building modules in Nemesis

Posted: 20 May 2020, 08:48
by M. Eerie
New approach...

This one is intended for base modules only.

I'd appreciate some feedback. Please, let me know if it works for you.

LAST UPDATED: Jun 3, 2020

Code: Select all

#!/bin/bash
# Update NEMESIS base modules ( $PORTDIR/base )
# <https://forum.porteus.org> 

function sqtype() { unsquashfs -s $@ | awk 'NR==4 {printf " " $2 " "}; NR==5 {print}'; }; export -f sqtype
#for mod in $MODDIR/*; do printf "$(basename $mod)\t"; sqtype $mod; done
function lssq() { unsquashfs -ll "$1" | sed 's/squashfs-root//g'; }; export -f lssq

HERE="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
warn() { echo; echo "$(tput setaf 1) $1 $(tput sgr0)"; echo; }
confirm() { echo; tput setaf 6; read -n1 -rsp "$1"; tput sgr0; echo -e "\n"; [[ $REPLY = [Yy] ]]; }

[ ! $(grep -o 'base' /proc/cmdline) ] && echo "Please init Nemesis with **Always Fresh** boot paramaters" && exit 0;
[ ! $EUID != 0 ] || sudo $(basename $0)

the_end() {
	if [ $? == 0 ]; then
		dir2xzm /tmp/LOCALES /tmp/LOCALES.xzm
		mv /tmp/'*.xzm' $HERE
		cd /tmp; rm -r LOCALES $outMOD >/dev/null 2>&1
		echo -ne "  All Done! \n Your files are here: $HERE"
	else 
		warn " Aborted. Exit code: $?"
	fi
	unset patches HERE outMOD
	eval "exit $?"
}
trap the_end SIGHUP SIGINT SIGTERM

fixPERMS() {
	chmod -f 750 $1/root #$1/etc/sudoers.d/ $1/etc/openvpn/client/ $1/etc/openvpn/server/
	chmod -f 755 $1/etc $1/etc/local.d $1/usr $1/usr/local $1/usr/local/bin
	find $1/usr/share/icons -type d -exec chmod 755 {} + >/dev/null 2>&1
	find $1/usr/share/icons -type f -exec chmod 644 {} + >/dev/null 2>&1
}

prepareSYS() {
fixPERMS ""

[ $(pacman -Qu | wc -l) == 0 ] || {
	setup-pman # WORKAROUND In case setup-pman fails --> "Please ensure you have a connection and try again. Exit code: 1"
	if [ $? == 1 ]; then
		pacman-db-upgrade && pacman -Sy
		haveged; sleep 3; pacman-key --init; pacman-key --populate archlinux artix; pkill haveged; update-ca-trust;
	fi
	pacman -Syyu --overwrite '*'
	touch /tmp/dbDONE ### avoid iterations
	}
	echo "  System updated!"; echo
}

updateBASE() {
	[ -f /tmp/dbDONE ] || prepareSYS
	[ ! -d /tmp/LOCALES ] && mkdir -p /tmp/LOCALES
	PS3=$'\n'$'\e[00;36mPlease, select base module to update or exit (1-5): \e[0m'
	select outMOD in core gui xtra xfce exit
	do
		patches=""
		case $outMOD in
			core) patches="{archlinux,artix}-{keyring,mirrorlist} {lib,}p11-kit gnupg file open{sysusers,tmpfiles} dbus{,-openrc} ca-certificates";;
			 gui) ;;
			xtra) patches="srt vmaf";; #TODO Can be used to interact with the user and add new deps.
			xfce) ;;
			exit*) the_end;;
		esac
	outMOD="$(basename $(ls -1 $PORTDIR/base | grep -i $outMOD) | cut -d- -f-2)"
	break
	done

cd /tmp
	unsquashfs -f -d $outMOD $PORTDIR/base/$outMOD*
	fixPERMS "$outMOD"

### PATCH CORE MODULE. TRICKY ONE AND NOT 100% WORKING AS post transactions hooks (eudev, openrc...) need to run in real root environment, (mountpoints, etc.)
	if [ $outMOD == "001-core" ]; then
		pacman -Rddr $outMOD {archlinux,artix}-{keyring,mirrorlist}
		rm -r $outMOD/etc/pacman.d/gnupg
		pacman-key --gpgdir $outMOD/etc/pacman.d/gnupg --init
		pacman-key --gpgdir $outMOD/etc/pacman.d/gnupg --populate archlinux artix
	fi

	cp -au --parents /var/lib/pacman/{sync/,local/ALPM_DB_VERSION} $outMOD
	pacman -Qqnr $outMOD | pacman --root $outMOD -Sdd - --overwrite '*'
	# UNDO previous patch / apply new deps
	[ -z "$patches" ] || eval "pacman -Sddr $outMOD $patches"

cd $outMOD
	find usr/share/locale \( -name 'e[ns]_[EU]S' -o -name 'e[ns]' -o -iname 'locale.alias' \) -exec cp -au --parents {} /tmp/LOCALES \;
	rm -r usr/share/{doc,gir-1.0,gtk-doc,info,licenses,locale,man} usr/include usr/lib/{pkgconfig,systemd} tmp boot >/dev/null 2>&1
	rm -r var/lib/pacman/{sync/,local/ALPM_DB_VERSION} >/dev/null 2>&1
	rm etc/ld.so.cache etc/sudo_logsrvd.conf etc/shadow.pacnew etc/sudo.conf

cd /tmp
	mv $outMOD/var/log/pacman.log $HERE/$outMOD-pacman.log >/dev/null 2>&1
	chown -R 0.0 $outMOD; [ -d $outMOD/home ] && chown -R 1000.1000 $outMOD/home >/dev/null 2>&1
	dir2xzm $outMOD "$outMOD"-$(date '+%Y%m%d').xzm
	rm -r $outMOD/
}

while confirm "Shall we update [Yy/*]? "; do updateBASE; done

the_end
NOTES:
  • Code refined. Pacman is such an amazing tool.
  • Some tasks involving creation of users accounts (sudo users, etc.) were skipped according to logs.
  • Additional note: Suggested updating method is from bottom to top. That is: xfce, then xtra, then gui, then core.
By now I've succeeded to get into Desktop with various methods.
When I have more time, I will do some tests and try to bring a final version...
I think that will be all for now.

If I'm not mistaken, the only pending thing is sudo account. I'll be back. B)

Cheers!

Updating/building modules in Nemesis

Posted: 28 May 2020, 10:25
by M. Eerie
Testing needed.

Anyone for the task?

Cheers!

Updating/building modules in Nemesis

Posted: 03 Jun 2020, 21:41
by M. Eerie
Testing still needed.

:roll:

Updating/building modules in Nemesis

Posted: 07 Dec 2020, 16:18
by M. Eerie
Solved. See here