[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Here is a place for your projects which are not officially supported by the Porteus Team. For example: your own kernel patched with extra features; desktops not included in the standard ISO like Gnome; base modules that are different than the standard ISO, etc...
i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#1 by i3slkiller » 05 Sep 2024, 16:25

WARNING! This script is during development and may be unstable, use at your own risk.

https://gitlab.com/i3slkiller/xzmbuild

Code: Select all

Usage: /mnt/vbox/xzmbuild/xzmbuild.sh [parameters] [path/to/XZMBUILD]

  -h, --help            show help
  -s, --src path        folder to save downloaded packages, sources
                        (default: the same as for XZMBUILD)
  -t, --tmp path        temporary folder (default: /tmp/xzmbuild)
  -o, --out path        XZM module folder output (default: /tmp)
  -p, --spkg path       folder for compiled packages (default: $pbtmpdir/spkg)
  -b, --sb path         temporary folder for SlackBuilds
                        (default: $pbtmpdir/slackbuild)
  --skip-download       don't download files
  --skip-verify         skip files verification
  --no-pgp-verify       disable PGP verification
  --no-build            don't build module
  -g, --generate-checksums  generates checksums of files to paste them
                            to XZMBUILD file and exits.
  -c, --comp xz|zstd    set compression algorithm for modules (default: xz)
  -d, --ask-makedepends  ask which makedepends module should be activated
                         if more than one match was found
                         (ignored if script started in non-interactive shell)
  -x, --find-makedepends-before path
                        find makedepends modules before searching in PORTDIR
                        (can be used multiple times)
                        WARNING! Searching in folder with large number of files
                        may take long time.
  -X, --find-makedepends-after path
                        find makedepends modules after searching in PORTDIR
                        (can be used multiple times)
                        WARNING! Searching in folder with large number of files
                        may take long time.
  -n, --no-find-makedepends-portdir
                        don't search makedepends modules in $PORTDIR/modules
                        and $PORTDIR/optional
  -v, --version         show script version
Test versions: https://archive.org/download/porteus-mo ... d/testing/

I'm writing this script to replace messy PorteusBuild scripts and make maintaining own modules easier.

Docs for this will be written later. Docs written in Pandoc Markdown are available in "man" folder in repo (you need to have pandoc to generate man pages from them)

As of 05.10.2024 xzmbuild branch has merged to master, so search example files here https://gitlab.com/i3slkiller/porteusbuilds
old example files: https://gitlab.com/i3slkiller/porteusbu ... rteus/5.01

Expect breaking changes!

10.09.2024 - renamed from PRTBUILD [#8]
29.09.2024 - "almost" finished writing docs, a "few" more fixes are needed
04.10.2024 - added link to test versions
05.10.2024 - updated example files link
Last edited by i3slkiller on 05 Oct 2024, 09:51, edited 6 times in total.

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 4123
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#2 by ncmprhnsbl » 05 Sep 2024, 23:53

very nice :)
at first glance, this looks like an arch style approach, analogous to (arch's) makepkg and PKGBUILD.
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

burdi01
Shogun
Shogun
Posts: 216
Joined: 18 Aug 2013, 12:09
Distribution: Slackware PartedMagic Xubuntu
Location: The Netherlands

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#3 by burdi01 » 07 Sep 2024, 10:15

Crux? :shock:

User avatar
Ed_P
Contributor
Contributor
Posts: 8653
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#4 by Ed_P » 07 Sep 2024, 22:34

i3slkiller wrote:
05 Sep 2024, 16:25
I'm writing this script to replace messy PorteusBuild scripts and make maintaining own modules easier.
Certainly more evolved than my mdmod scripts:

Code: Select all

#!/bin/bash

# Save changes as a mod file  

# https://forum.porteus.org/viewtopic.php?f=81&t=5981&start=15#p46643

MOD=myKeePasschanges   # mychanges-$(date +%Y%m%d)
TO=Optional
SYSTM=

# http://forum.porteus.org/viewtopic.php?f=53&t=3801&start=30#p28472
BOOT=`grep -A1 "Porteus data found in:" /var/log/porteus-livedbg|tail -n1|sed 's^//^/^g'`
DRV=${BOOT:5:4}
if [ "$DRV" == "isol" ]; then
   ISOBOOT=`grep -A1 "//porteus" /var/log/porteus-livedbg|tail -n2|sed 's^//^/^g'`
   SYSTM="${ISOBOOT:10:10}/"
   DRV=${ISOBOOT:5:4}
fi
if [ "$DRV" == "nvme" ]; then
   SYSTM="${ISOBOOT:15:10}/"
   DRV=${ISOBOOT:5:9}
fi

CHANGES=/mnt/$DRV/$SYSTM$TO/$MOD
#echo BOOT: $BOOT & echo ISOBOOT: $ISOBOOT & echo DRV: $DRV & echo SYSTM: $SYSTM & echo CHANGES: $CHANGES & read

if [ "$1" == "r" ] && [ "$2" != "" ] ; then
   CHANGES="$2"
fi

if [ `whoami` != "root" ]; then
   echo toor | sudo -S  echo "******"
   sudo sh $0 $1 $2
   exit
fi
echo -e "\033[0m"; echo -en "\033]0;Make a Mod\a" 

echo "***** Must run AFTER KeePass has run & used the .kdbx file. *****"

if [ -d /tmp/mod/ ]; then
   rm -rf /tmp/mod
fi
mkdir -p /tmp/mod

# copy modified files

#                  -- KeePass Open Recent file --
cp -a --parents    /home/guest/.config/KeePass/*                     /tmp/mod/

du -h    /tmp/mod/   | grep "[0-9]M"
echo
echo Changed files copied. Press Enter to create $MOD, Ctrl+C to exit.
echo

if [ -f $CHANGES*.xzm ]; then
   CF=`ls -lght --time-style long-iso $CHANGES*.xzm | awk '{print $5,$6}'`
   if [ ${CF:11:2} -lt 12 ]; then
      CF=$CF"am"
   else
      PM=$((${CF:11:2} - 12))
      if [ $PM == 0 ]; then PM=12; fi
      CF=${CF:0:11}$PM${CF:13}"pm"
   fi
   CFS=`ls -lght --time-style long-iso $CHANGES.xzm | awk '{print $4}'`
   echo Current file:  $CFS $CF
fi
read

dir2xzm  /tmp/mod /tmp/$MOD.xzm   &&   rm -rf /tmp/mod

echo 
ls -sh   /tmp/*.xzm

DIR () {
ls -lght --time-style long-iso $1 | awk '{print $5,$6,$4,$7}'
}

echo
echo Press Enter to copy /tmp/$MOD.xzm to $SYSTEM$TO, Ctrl+C to exit.
DIR     /tmp/$MOD.xzm 
if [ -f $CHANGES*.xzm ]; then
   DIR     $CHANGES*.xzm
fi 
read
if [ -f $CHANGES*.xzm ]; then
   mv      $CHANGES*.xzm $CHANGES.xyzm
fi
cp      /tmp/$MOD.xzm  /mnt/$DRV/$SYSTM$TO
sleep 1
rm      /tmp/$MOD.xzm
echo Finished!
sleep 1
burdi01 wrote:
07 Sep 2024, 10:15
Crux? :shock:
:%)

burdi01
Shogun
Shogun
Posts: 216
Joined: 18 Aug 2013, 12:09
Distribution: Slackware PartedMagic Xubuntu
Location: The Netherlands

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#5 by burdi01 » 08 Sep 2024, 09:48

When googling wtf these prtbuild files are I got the impression that they are native Crux packages.
B)

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#6 by i3slkiller » 08 Sep 2024, 09:54

burdi01 wrote:
08 Sep 2024, 09:48
When googling wtf prtbuild files are I got the impression that they are the the native Crux packages.
Hmm, can you give me example of such files?

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 4123
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#7 by ncmprhnsbl » 08 Sep 2024, 12:23

i3slkiller wrote:
08 Sep 2024, 09:54
Hmm, can you give me example of such files?
https://git.crux.nu/ports/core/src/bran ... ls/Pkgfile
the point in common with this and Arch's PKGBUILD and void's template is that you have a file that provides basic info (name, version, compilation function etc) that a primary build script/program uses to produce a package.
(arch: makepkg crux: pkgmk void: xbps-src (which goes even further by being it's own clean chroot))
in contrast with the slackware .SlackBuild which is just a script that compiles and produces a package.
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#8 by i3slkiller » 09 Sep 2024, 08:39

ok, I found it https://git.crux.nu/tools/prt-utils and I see that current name may be confusing for crux users, so maybe I will rename it to "xzmbuild". How do you think about that?

User avatar
Ed_P
Contributor
Contributor
Posts: 8653
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

[WIP] prtbuild - tool for creating Porteus modules from PRTBUILD files

Post#9 by Ed_P » 09 Sep 2024, 15:48

i3slkiller wrote:
09 Sep 2024, 08:39
maybe I will rename it to "xzmbuild". How do you think about that?
:good:

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#10 by i3slkiller » 10 Sep 2024, 06:51

Renamed to "xzmbuild", updated first post

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#11 by i3slkiller » 29 Sep 2024, 09:24

xzmbuild docs written in Pandoc Markdown are now available in "man" folder in repo (you need to have pandoc to generate man pages from them).

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#12 by i3slkiller » 04 Oct 2024, 08:14

First testing version of xzmbuild is released, let me know about your comments about it and any errors you find https://archive.org/download/porteus-mo ... arch-1.xzm
From now I will start build own modules using this tool.

Code: Select all

8baaa75ca80974e534ef399d41522fe82b32d9850a603cac2edae22ef1145f27  xzmbuild-1_05829c9b18cde523-noarch-1.xzm
05829c9b18cde523 string is truncated SHA512 hash of xzmbuild.sh script.

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#13 by i3slkiller » 05 Oct 2024, 10:43

New testing version of xzmbuild was released, it fixes failure of downloading keys from keyserver by gnupg2 (set custom homedir instead keyring file)
https://archive.org/download/porteus-mo ... arch-1.xzm

Code: Select all

2fd4fbcc96a16cb09ecf5f12b4a4c345060bd989191f6ae6a3d6e9d18e475881  xzmbuild-1_ac726a75c165d684-noarch-1.xzm

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#14 by i3slkiller » 05 Oct 2024, 12:25

New testing version of xzmbuild was released, it fixes python venv scripts shebangs, deletes activate venv files and fixed INSTALLPYROOT variable typo when xzmname is array which caused creating venv outside INSTALLPYROOT
https://archive.org/download/porteus-mo ... arch-1.xzm

Code: Select all

6c8228e236781ce0afd9989d37d23dc8688ed01ae6c412a13952836f902e8b6e  xzmbuild-1_77e4284e40d1b87f-noarch-1.xzm

i3slkiller
Contributor
Contributor
Posts: 120
Joined: 03 Feb 2020, 18:51
Distribution: Porteus XFCE v5.0rc2 x86_64
Location: Poland

[WIP] xzmbuild - tool for creating Porteus modules from XZMBUILD files

Post#15 by i3slkiller » 31 Oct 2024, 12:38

Has anyone already tried to write own XZMBUILD scripts and can share its conclusion about it?

I wonder if this tool should:
- delimit by colons instead of commas in slack_prebuilt_packages, slack_builds, *checksums arrays:

Code: Select all

slackware,kde/extra-cmake-modules,5.90.0-1,native
slackware:kde/extra-cmake-modules:5.90.0-1:native

sha512,blablabla
sha512:blablabla

pgp,asc
pgp:asc
although in *checksums arrays there may be

Code: Select all

pgp,https://blabla.bla/bla/bla.asc
so it will be look like below :hmmm:

Code: Select all

pgp:https://blabla.bla/bla/bla.asc
- handle slackbuilds building multiple packages (applies to some slackware slackbuilds) (to think about how to do it)
- allow define distro name and version for which xzmbuild was written, so in other distro version than specified script will stop even before loading repository info files, unless --force specified
- rename variables and arrays:
-- xzmname -> name?
-- xzmver -> version?
-- xzmrel -> xzmbuild/build?
-- slack_prebuilt_packages -> slack_packages
-- slack_builds_additional_parameters -> slack_builds_parameters
-- ???
- and specify multiple checksums for one file (eg. sha512, b2, sm3, in case of discovering weakness of some of hash algorithm even if one checksum of faked file will be correct, others will not [look at md5, sha1])
- EDIT 01.11.2024: split xzmbuild.sh to multiple files like in split_xzmbuild_sh branch

How do you think? Apply these changes or keep as-is?

Post Reply