applying a script to get PACKAGES.TXT for ponce repo

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#1 by francois » 01 Mar 2023, 20:03

When the PACKAGES.TXT for a repository is not available for a given set of packages, we read in Faq 17 for slapt-get that one can use a script:
https://github.com/jaos/slapt-get/blob/main/FAQ

I would like to apply the procedure to ponce directory which does not include the dependencies.

It is said that:

17. How do I create my own package source?

Within slapt-getrc, change your SOURCE= lines to point to your package
source. This might be a local source using file:// URLs, or a publicly
available source.

For example, you could have an official and a local source like:
SOURCE=ftp://ftp.slackware.no/pub/linux/slackw ... kware-9.1/
SOURCE=file:///usr/src/local_pkg_repository/

This local directory must have the PACKAGES.TXT and CHECKSUMS.md5 files
present. This could be a mounted Slackware release CDROM, or a custom
repository.

The CHECKSUMS.md5 file can be generated with find:
rm CHECKSUMS.md5; find . -name '*.tgz' -exec md5sum {} >> CHECKSUMS.MD5 \;


PACKAGES.TXT package database that Patrick Volkerding provides along with his
packages. slapt-get uses this file by extending it with optional extra fields.
This information is stored within the package simply as a means of easy
transport, to later be parsed into a PACKAGES.TXT. For example, the entry
for man within PACKAGES.TXT looks like:

Code: Select all

   PACKAGE NAME:  man-1.5l-i386-1.tgz
   PACKAGE LOCATION:  ./slackware/ap
   PACKAGE SIZE (compressed):  166 K
   PACKAGE SIZE (uncompressed):  390 K
   PACKAGE DESCRIPTION:
   man: man (format and display the on-line manual pages)
It is extended like so:

Code: Select all

 PACKAGE NAME:  man-1.5l-i386-1.tgz
   PACKAGE MIRROR:  http://www.slackware.at/data/slackware-9.1/
   PACKAGE LOCATION:  ./slackware/ap
   PACKAGE SIZE (compressed):  166 K
   PACKAGE SIZE (uncompressed):  390 K
   PACKAGE REQUIRED: groff >= 1.56-noarch-1,man-pages | man-pages-de
   PACKAGE CONFLICTS:  
   PACKAGE SUGGESTS:  
   PACKAGE DESCRIPTION:
   man: man (format and display the on-line manual pages)
The REQUIRED line is an addition supported by slapt-get, along with CONFLICTS
and SUGGESTS. The meta files supporting dependencies, conflicts, and
suggestions are within the packages inside the ./install/ directory. The
REQUIRED information is stored in the slack-required file. The CONFLICTS
information is stored within the slack-conflicts file. The SUGGESTS
information is stored in the slack-suggests file.

See FAQ #19 for a breakdown
of the structure of REQUIRED, FAQ #31 for CONFLICTS, and FAQ #44 for SUGGESTS.
MIRROR is an optional location so that the packages can be hosted elsewhere
(this is the internal representation slapt-get uses after caching the package
date from the remote package source).

This information is added to the PACKAGES.TXT file within the package repository
by the provider of the packages.

Here is the script:

Code: Select all

 ### BEGIN SCRIPT
    #!/bin/sh
    #DL_URL=http://your_remove_pkg_host.tld/packages/
    # This program is free software; you can redistribute it and/or modify
    # it under the terms of the GNU General Public License as published by
    # the Free Software Foundation; either version 2 of the License, or
    # any later version.

    function gen_packages_txt {
	    echo '' > PACKAGES.TXT
	    find . -type f -name '*.meta' -exec cat {} \; >> PACKAGES.TXT
      cat PACKAGES.TXT | gzip -9 -c - > PACKAGES.TXT.gz
    }

    function gen_md5_checksums {
	    echo '' > CHECKSUMS.md5
	    find . -type f -regextype posix-egrep -regex '.*\.[tgblzikx]+$' -exec md5sum {} \; >> CHECKSUMS.md5
      cat CHECKSUMS.md5 | gzip -9 -c - > CHECKSUMS.md5.gz
    }

    function gen_meta {
	    if [ ! -f $1 ]; then
		    echo "File not found: $1"
		    exit 1;
	    fi
			if [ "`echo $1|grep -E '(.*{1,})\-(.*[\.\-].*[\.\-].*).[tgblzikx]{2,}[ ]{0,}$'`" == "" ]; then
				return;
			fi

      PKGEXT=${1##*.}
      case $PKGEXT in
        tgz) DECOMPRESS=gzip ;;
        tbz) DECOMPRESS=bzip2 ;;
        tlz) DECOMPRESS=lzma ;;
        txz) DECOMPRESS=xz ;;
      esac

	    NAME=$(echo $1|sed -re "s/(.*\/)(.*.$PKGEXT)$/\2/")
	    LOCATION=$(echo $1|sed -re "s/(.*)\/(.*.$PKGEXT)$/\1/")
	    SIZE=$(du -bk $1 | awk '{print $1}')
	    USIZE=$(expr $(cat $1 | $DECOMPRESS -dc | wc -c) / 1024)
	    REQUIRED=$($DECOMPRESS -dc $1 | tar -xO install/slack-required 2>/dev/null|xargs -r -iZ echo -n "Z,"|sed -e "s/,$//")
	    CONFLICTS=$($DECOMPRESS -dc $1 | tar -xO install/slack-conflicts 2>/dev/null|xargs -r -iZ echo -n "Z,"|sed -e "s/,$//")
	    SUGGESTS=$($DECOMPRESS -dc $1 | tar -xO install/slack-suggests 2>/dev/null|xargs -r )
	    METAFILE=${NAME%$PKGEXT}meta
	    echo "PACKAGE NAME:  $NAME" > $LOCATION/$METAFILE
	    if [ -n "$DL_URL" ]; then
	    	echo "PACKAGE MIRROR:  $DL_URL" >> $LOCATION/$METAFILE
	    fi
	    echo "PACKAGE LOCATION:  $LOCATION" >> $LOCATION/$METAFILE
	    echo "PACKAGE SIZE (compressed):  $SIZE K" >> $LOCATION/$METAFILE
	    echo "PACKAGE SIZE (uncompressed):  $USIZE K" >> $LOCATION/$METAFILE
	    echo "PACKAGE REQUIRED:  $REQUIRED" >> $LOCATION/$METAFILE
	    echo "PACKAGE CONFLICTS:  $CONFLICTS" >> $LOCATION/$METAFILE
	    echo "PACKAGE SUGGESTS:  $SUGGESTS" >> $LOCATION/$METAFILE
	    echo "PACKAGE DESCRIPTION:" >> $LOCATION/$METAFILE
	    $DECOMPRESS -dc $1 | tar -xO install/slack-desc |grep -E '\w+\:'|grep -v '^#' >> $LOCATION/$METAFILE
	    echo "" >> $LOCATION/$METAFILE
    }

    case "$1" in
	    pkg)
		    if [ -n "$2" ]; then
			    gen_meta $2
		    else
			    echo "$0 [pkg [file]|all|new|PACKAGESTXT|MD5]"
		    fi
	    ;;
	    all)
		    for pkg in `find . -type f -regex '.*\.[tgblzikx]+$' -print`
		    do
			    gen_meta $pkg
		    done
		    $0 PACKAGESTXT
		    $0 MD5
	    ;;
	    new)
		    for pkg in `find . -type f -regex '.*\.[tgblzikx]+$' -print`
		    do
			    if [ ! -f ${pkg%${pkg##*.}}meta ]; then
				    gen_meta $pkg
			    fi
		    done
	    ;;
	    PACKAGESTXT)
		    gen_packages_txt
	    ;;
	    MD5)
		    gen_md5_checksums
	    ;;
	    *)
		    echo "$0 [pkg [file]|all|new|PACKAGESTXT|MD5]"
	    ;;
    esac
 ### END SCRIPT
Prendre son temps, profiter de celui qui passe.

User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#2 by francois » 01 Mar 2023, 20:18

I understand that the script will create a new PACKAGES.TXT file including the dependencies.

Here is what I have done:
- installed the script in a working directory and name it PACKAGES.TXT-script-to-include-dependencies.sh, gave exec permissions
- in the script I have inserted line 4 to provide ponce repository , thus from:

Code: Select all

 ### BEGIN SCRIPT
    #!/bin/sh
    #DL_URL=http://your_remove_pkg_host.tld/packages/
    # This program is free software; you can redistribute it and/or modify
to

Code: Select all

### BEGIN SCRIPT
    #!/bin/sh
    #DL_URL=http://your_remove_pkg_host.tld/packages/
	DL_URL=https://ponce.cc/slackware/slackware64-15.0/packages/
    # This program is free software; you can redistribute it and/or modify
This does not yield the PACKAGES.TXT file including the dependencies.

Any cue?
Prendre son temps, profiter de celui qui passe.

User avatar
Ed_P
Contributor
Contributor
Posts: 8369
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

applying a script to get PACKAGES.TXT for ponce repo

Post#3 by Ed_P » 01 Mar 2023, 22:09

Have you tested it for the original URL?
Ed

User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#4 by francois » 01 Mar 2023, 22:45

Yes withh the change in the file exactly as explained in my previous post with
DL_URL=https://ponce.cc/slackware/slackware64-15.0/packages/

Output:

Code: Select all

packages.txt-transformation.sh*
slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh*
root@porteus:~/Documents/WORKING-DIR# ./slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh*
./slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh [pkg [file]|all|new|PACKAGESTXT|MD5]
??
Prendre son temps, profiter de celui qui passe.

User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#5 by francois » 01 Mar 2023, 22:48

Maybe I should create local directory as the instructions stipulate:

SOURCE=file:///usr/src/local_pkg_repository/

This local directory must have the PACKAGES.TXT and CHECKSUMS.md5 files
present. This could be a mounted Slackware release CDROM, or a custom
repository.
Prendre son temps, profiter de celui qui passe.

User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#6 by francois » 02 Mar 2023, 00:00

I have tried within the specified directory with only one package from ponce, that is abiword, in addition to the script of faq 17 to get PACKAGES.TXT with dependencies produced. And as I got no result, I have extracted abiword-3.0.5-x86_64-1ponce.xzm to directory. I had also put abiword-3.0.5-x86_64-1ponce.txz in that directory.

Code: Select all

root@porteus:/usr/src/local_pkg_repository# ls
CHECKSUMS.MD5
abiword-3.0.5-x86_64-1ponce/
abiword-3.0.5-x86_64-1ponce.txz
abiword-3.0.5-x86_64-1ponce.xzm
packages.txt-transformation.sh*
slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh*
root@porteus:/usr/src/local_pkg_repository# 
and then picked up the info from
/usr/src/local_pkg_repository/abiword-3.0.5-x86_64-1ponce/usr/doc/abiword-3.0.5/
abiword-3.0.5# ls
AUTHORS BiDiReadme.txt COPYRIGHT.TXT README
BUILD.TXT COPYING NEWS abiword.SlackBuild
and transferred into
root@porteus:/usr/src/local_pkg_repository#

Then:

Code: Select all

root@porteus:/usr/src/local_pkg_repository# rm CHECKSUMS.md5; find . -name '*.txz' -exec md5sum {} >> CHECKSUMS.MD5 \; # here I changed tgz for txz and we get:
root@porteus:/usr/src/local_pkg_repository# ls
AUTHORS         README
BUILD.TXT       abiword-3.0.5-x86_64-1ponce/
BiDiReadme.txt  abiword-3.0.5-x86_64-1ponce.txz
CHECKSUMS.MD5   abiword-3.0.5-x86_64-1ponce.xzm
COPYING         abiword.SlackBuild
COPYRIGHT.TXT   packages.txt-transformation.sh*
NEWS            slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh*

Code: Select all

root@porteus:/usr/src/local_pkg_repository/abiword-3.0.5-x86_64-1ponce/usr/doc/abiword-3.0.5# ./slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh
bash: ./slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh: No such file or directory
root@porteus:/usr/src/local_pkg_repository/abiword-3.0.5-x86_64-1ponce/usr/doc/abiword-3.0.5# 
No results:
biword-3.0.5# ./slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh
bash: ./slapt-get-faq17-PACKAGES.TXT-with-dependencies.sh: No such file or directory
root@porteus:/usr/src/local_pkg_repository/abiword-3.0.5-x86_64-1ponce/usr/doc/abiword-3.0.5#
Prendre son temps, profiter de celui qui passe.

User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#7 by francois » 02 Mar 2023, 00:09

I forgot to say that the second script has been handed directly by Patrick Volkerding.
https://www.linuxquestions.org/question ... 175722428/
I have called it
packages.txt-transformation.sh*

The other guys that answered me are well implacate slackware community members, Didier Spaier and Brains. They seemed to disregard Patrick suggestion. I wonder why.

Also, I have contacted ponce to see if he wouldn't provide the PACKAGES.TXT with dependencies:
https://github.com/Ponce/slackbuilds/is ... 1448618477

I have contacted the maintainer of slapt-get Jason Woodward (Jaos):
https://github.com/jaos/gslapt/issues/5 ... 1439336706
Essentially, he told me that it was possible to use a download only function as muche for slapt-get than for gslapt.

Thanks.
Prendre son temps, profiter de celui qui passe.

User avatar
francois
Contributor
Contributor
Posts: 6435
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

applying a script to get PACKAGES.TXT for ponce repo

Post#8 by francois » 03 Mar 2023, 15:31

I found better than the script of Volkerding. Here is a detailed description on setting up a local repository with dependency support:
https://docs.salixos.org/wiki/How_to_cr ... cy_support

Feel free to set one up on your linux box too.

Thanks to jaos the maintainer of slapt-get.
Prendre son temps, profiter de celui qui passe.

Post Reply