[RELEASE] src2pkg-2.9

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
amigo
White ninja
White ninja
Posts: 15
Joined: 26 Aug 2013, 14:14
Distribution: KISS
Location: Dreieich

[RELEASE] src2pkg-2.9

Post#1 by amigo » 26 Aug 2013, 14:21

I've just released a new version of src2pkg(2.9), which fixes a couple of minor bugs when dealing with filenames which contain spaces.

The ChangeLog is here:
http://distro.ibiblio.org/amigolinux/do ... /ChangeLog

The installable package for Porteus is here:
http://distro.ibiblio.org/amigolinux/do ... arch-1.txz

A new option has been added: Setting the PKG_EXCLUDES variable lets you specify a comma-separated list of files, links or dirs to exclude from a package. The variable can be passed as an environmental variable, placed in a *.src2pkg build script, in the src2pkg.conf file or from the command line using the option: '--excludes=??,??' Thanks to Chris Lee <c.lee111@gmail.com> for requesting this feature, which now that I think about it, should have been there all along.

If you are interested in seeing what Slackers think about src2pkg, you can follow the announcement thread here:
https://www.linuxquestions.org/question ... 175474766/

User avatar
fanthom
Site Admin
Site Admin
Posts: 4567
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland, currently - Cork, IE
Contact:

Re: [RELEASE] src2pkg-2.9

Post#2 by fanthom » 26 Aug 2013, 16:42

certainly useful utility,

thanks for sharing amigo.
Please add [Solved] to your thread title if the solution was found.

amigo
White ninja
White ninja
Posts: 15
Joined: 26 Aug 2013, 14:14
Distribution: KISS
Location: Dreieich

Re: [RELEASE] src2pkg-2.9

Post#3 by amigo » 26 Aug 2013, 18:15

I'd certainly like to work with you to perfect an 'extension' for src2pkg which would create your modules for you. Actually, big_bass shared some bits of code with me which are nearly ready to try.

I have read all the comments about src2pkg which have been made here. Most of them can be addresses easily enough -things like creating smaller packages. There are options for compressing binaries, minimizing docs, splitting packages into devel, doc, nls sub-packages, etc. A good way to handle these would be to come up with a pre-configured src2pkg.conf file. While it would be possible to create lots of extra code for generating modules for porteus *instead* of a normal *.txz package, I think that a simple extension would be a better solution. src2pkg extensions are simply one or more script snippets which can be run before and/or after each of the src2pkg build steps -so there is lots of room for whatever is needed.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: [RELEASE] src2pkg-2.9

Post#4 by francois » 27 Aug 2013, 01:11

Src2pkg is really interesting. This is the related thread that bigbass wrote on src2pkg and its use thru sbopkg, though src2pkg could be used alone:
viewtopic.php?f=75&t=1041

The following is a little outdated review of src2pkg. Nonetheless, it is quite positive:
http://archive09.linux.com/feature/121499

To work on porteus 2.1, the devel.xzm module is required:
http://dl.porteus.org/x86_64/current/modules/

Will src2pkg do a better job than sbopkg alone?
Last edited by francois on 25 Dec 2013, 21:41, edited 5 times in total.
Voltaire: Le mieux est l'ennemi du bien.

amigo
White ninja
White ninja
Posts: 15
Joined: 26 Aug 2013, 14:14
Distribution: KISS
Location: Dreieich

Re: [RELEASE] src2pkg-2.9

Post#5 by amigo » 27 Aug 2013, 05:21

src2pkg can often build packages from sources even when no 'recipe' or build script is used. It remains unique in this aspect. Of course, using a *.src2pkg build script provides a record of how the package was built and gives you full access to all options, tricks, etc -including being able to skip steps or add arbitrary code.

src2pkg includes lots and lots of checks and fixes on the package content to ensure completeness and sanity. It also has any 'advanced' instructions for special package needs -like making things smaller, generating dependency info, etc. I maintain a distro which is built entirely using src2pkg build scripts.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: [RELEASE] src2pkg-2.9

Post#6 by francois » 27 Aug 2013, 15:27

1) Error message trying to install scr2pkg on porteus v 2.1 x86_64.

Code: Select all

root@porteus:~# src2pkg --setup
  Notice - Creating src2pkg-helpers:
  src2pkg uses a shared library and a few programs
  when creating packages. For best compatibility,
  these binaries will be compiled on your system.
  They are then installed in a private directory.
  When done, src2pkg is ready for use.

  TEMP_DIR=/usr/src/src2pkg/builds/src2pkg-helpers
  Starting build in 5 seconds
Unpacking sources - OK
Creating libsentry - Ooops! Can't live without it...

2) Uses of src@pkg.
From the information that you provide in your documentation, I understand that I could use src2pkg to convert a .deb or .rpm program into some form of slackbuild that will call all the necessary dependencies and then be able to build a slackware package.

For example, I could convert:
https://code.google.com/p/pychess/downl ... &q=pychess
to pychess_0.10.1-1_all.txz all dependencies included.
http://katrine.lpi.ru/kalenkov/fedora/1 ... 8.i686.rpm
to scid-4.4-1.fc18.i686.txz all dependencies included.

Is that true?
Voltaire: Le mieux est l'ennemi du bien.

amigo
White ninja
White ninja
Posts: 15
Joined: 26 Aug 2013, 14:14
Distribution: KISS
Location: Dreieich

Re: [RELEASE] src2pkg-2.9

Post#7 by amigo » 27 Aug 2013, 17:49

1. Do you have a pretty complete development environment on that machine? You need gcc, glibc-headers, make, autoconf & Co., etc.
AFAIK, src2pkg build fine on a normal Slackware-64 box. Are you able to compile other/most sources normally?


2. No, you haven't understood. src2pkg can be used to convert from one package type to another -but that is certainly not what I recommend. In 'normal' operation, src2pkg is given the name of a source tarball or an already-created NAME.src2pkg build script. src2pkg then prepares all temp dirs, unpacks the sources, configures and compiles them. Then, it runs the appropriate command, like `make install DESTDIR=$PKG_DIR`, in order to 'fake' the installation of the files into the PKG_DIR where the package is being assembled. Then, it does a bunch of sanitizing steps and creates the final package. It does not download any dependencies -it figures out which packages this new package depends on -from the existing package database. For some package formats then this dependency info is included in the package -other tools are used to deal with that information. However, src2pkg can be used to build and install-as-you-go any number of packages in series.

What I like most about src2pkg is the simple clean build scripts -which src2pkg can even generate for you. The 'clean' content and formatting make it really easy to see when you've 'intervened' in the script for special needs. Just as with SlackBuild scripts, you can code anything in there that is needed.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: [RELEASE] src2pkg-2.9

Post#8 by francois » 27 Aug 2013, 21:36

1) Ok, scr2pkg is working now with the addition of the porteus devel.xzm module (edited my prior post to include this dependency):
http://dl.porteus.org/x86_64/current/modules/

2) Should I understand that src2pkg will be usefull to build the chess game with the source package called scid vs pc, thru:
http://sourceforge.net/projects/scidvsp ... ecommended

That it will identify dependencies, that I will then have to install manually these dependencies to get the package done? Reflecting on it, I do not think so. The checking of dependencies will have to be done anyway. There is no dependency resolution with slackware! I take my dreams for some reality. :ROFL:

Anyway, I am going to have a try on building some packages in the next few days and I will report my impressions :)
Last edited by francois on 28 Aug 2013, 16:46, edited 1 time in total.
Voltaire: Le mieux est l'ennemi du bien.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: [RELEASE] src2pkg-2.9

Post#9 by francois » 28 Aug 2013, 13:55

Modified the dependencies needed for src2pkg in previous post. Only the devel.xzm module needed :)
Voltaire: Le mieux est l'ennemi du bien.

amigo
White ninja
White ninja
Posts: 15
Joined: 26 Aug 2013, 14:14
Distribution: KISS
Location: Dreieich

Re: [RELEASE] src2pkg-2.9

Post#10 by amigo » 01 Sep 2013, 18:15

src2pkg can only find out the dependencies once it has successfully built the package. The person creating the packages still has to figure out, or choose, what is needed to be able to build it. Actually, the routines may not even be working so great for *.tgz packages -I've concentrated the implementation of those features into the KISS *.tpkg packages, which are actually *.txz packages, but with some dir structure changes. Getting full advantage of the depends info needs a few scripts written for the purpose, plus a package manager which takes them into account. A slack-requires file can be included in a *.txz package, but it wont make it through the installation -installpkg will simply remove it. Since porteus has some mechanism for managing *.xzm modules, it wouldn't take much to have it use the depends info generated by src2pkg.

Still, src2pkg is always worth using -even when converting packages from another format- because of the numerous sanity checks(and optional corrections) run after assembling the package content and before creating the actual package. Also, if *.xzm modules were generated by a src2pkg extension, you can do anything needed or wanted there and ignore any final package created by src2pkg. While possible to get src2pkg to create modules *instead* of packages, I'm not keen on adding that directly to src2pkg -there are other possibilities -like creating AppDirs instead of packages...

For KISS, I have written my own package manager and database tools which provide *lots* of info -you can even derive a compile-order for all requirements of any program. However, I have not set implemented any automated dependency-resolution into the package manager -though it will not really be a big thing to do so.

The real trouble about having and using the package info is that it needs to be sane and complete -otherwise it is useless. Unfortunately the data which could be derived form a slackware package databse is not sane -in fact the packages themselves are not sane. I mean that they are not cleanly derived -you can never re-compile slackware from scratch and get the same result (binary-identical) as what is found in the repos. A great many of the packages were built using former toolchain versions and linked against former versions of libs. For KISS, I decided to simplify the dependency issue as much as possible in order to make things run smoother -instead of trying to indicate usable versions of dependencies with <= or >= logic. Instead, when src2pkg lists a dependency it uses the current installed version -after all that's the one that just got linked-to to create your package. If you then upgrade one of those depends, then your just-made package will still indicate that it depends on the former hard-wired version. I personally think this is more valuable information -it tells me exactly the version and build of the depends which were know to work at compilation time. Besides, implementing any <= or >= functionality means human intervention in determining the depends. Still, src2pkg does allow the packager to intervene when necessary. For instance, when there are non-binary dependencies or ones that can't be automatically determined -like the fact that 'man' needs 'groff'. For KISS, I can even determine the dependencies for shell scripts (and partially others) -I use a modified bash which can pick out all the programs called by a script.

xtudiux
Black ninja
Black ninja
Posts: 72
Joined: 07 Mar 2011, 06:01
Location: Philippines

Re: [RELEASE] src2pkg-2.9

Post#11 by xtudiux » 20 Sep 2013, 17:50

how do you add uxterm in src2pkg-dnd???

amigo
White ninja
White ninja
Posts: 15
Joined: 26 Aug 2013, 14:14
Distribution: KISS
Location: Dreieich

Re: [RELEASE] src2pkg-2.9

Post#12 by amigo » 23 Sep 2013, 16:26

Hmmm, you'd need to edit it to use uxterm instead of xterm -if uxterm supports the syntax used. Fine that you've found src2pkg-dnd -it is handy -mostly for having it write a script for you -so you don't have to spell out tarball-names. And it will work fine to build simple things which need no extra options. It isn't installed into your PATH by the src2pkg package, so your edited version won't get clobbered when upgrading src2pkg.

User avatar
fanthom
Site Admin
Site Admin
Posts: 4567
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland, currently - Cork, IE
Contact:

Re: [RELEASE] src2pkg-2.9

Post#13 by fanthom » 03 Dec 2013, 18:59

hello amigo,

i'm trying to pull your src2pkg to next porteus release but i'm having an issue with src2pkg-dnd which displays following message:

Code: Select all

Running src2pkg
Press <enter> to quit ...
/usr/bin/src2pkg-dnd: line 88: ./terminal-wrapper: No such file or directory
where can i find this 'terminal-wrapper'? shouldn't it be a part of the src2pkg package?

btw: was trying to contact with you on amigo@ibiblio.org but my email never reached the recipient.
Please add [Solved] to your thread title if the solution was found.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: [RELEASE] src2pkg-2.9

Post#14 by francois » 08 Dec 2013, 03:36

The installable package for Porteus is here:
http://distro.ibiblio.org/amigolinux/do ... arch-1.txz
The precedent link does not work no more, so use:
http://distro.ibiblio.org/amigolinux/download/src2pkg/

I thought it was not that evident to understand how to build from source src2pkg. But it seems that building from source is simply not evident. Reading that old article and the faq helps:
http://archive09.linux.com/feature/121499
http://www.src2pkg.net/faq

I found a few examples of the use of src2pkg or exercices that you could do with it to be accustomed to it:
http://www.src2pkg.net/quickstart
http://www.src2pkg.net/tutorial
http://archive09.linux.com/feature/121499

I have played around a little with src2pkg. It seems really interesting. Who else did so and what are your impressions? What packages have you built with src2pkg? What kind of exercices would you propose to get more familiar with this package builder? Please give us some example of src2pkg source building, the command line you used and the specific variables that you have set.
Voltaire: Le mieux est l'ennemi du bien.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: [RELEASE] src2pkg-2.9

Post#15 by francois » 09 Dec 2013, 03:46

An example under porteus:
1) using the deb file to build a slackware package:

Code: Select all

root@porteus:~/di# src2pkg -CWD stockfish_3.0.0+git20130508-2_i386.deb
...
...
earching for links in: stockfish-3.0.0+git20130508-i386-1 - None found
Package dependencies:
/lib/libc.so.6
/lib/libm.so.6
/lib/libpthread.so.0
/usr/lib/libgcc_s.so.1
/usr/lib/libstdc++.so.6
Creating package: stockfish-3.0.0+git20130508-i386-1.txz - Done
Package Creation - Successful! Package Location:
/root/di/stockfish-3.0.0+git20130508-i386-1.txz
root@porteus:~/di#
This seemed to work just fine. But I still have to test the thing.

However,
2) building from source I get:

Code: Select all

root@porteus:~/di# src2pkg -CWD stockfish-dd-src.zip
Found source archive: stockfish-dd-src.zip
Creating working directories:
   PKG_DIR=/root/di/stockfish-dd-src-i486-1
   SRC_DIR=/root/di/stockfish-dd-src-src-1
Unpacking source archive - Done
Correcting source permissions - Done
Checking for patches - None found
   Notice - The configuration files are in a subdirectory: src
Skipping configuration: Nothing to be done
Continuing - We found at least one Makefile
Compiling sources - Using: 'make'
Compiling has been - Successful!
Checking for Makefile rule: 'install' Okay
Checking support for DESTDIR (or similar) - Not found
   Notice - DESTDIR support not found. Falling back to JAIL install
Creating content in JAIL root - Using: 'make install'
FATAL! Running 'make install' has failed with error: 1 
Try using INSTALL_LINE 'make -i install'  Exiting...
root@porteus:~/di# 
Then trying to see what went wrong, I found some info in the readme.md file:

Code: Select all

root@porteus:~/di# cd /root/di/stockfish-dd-src-src-1/src/
root@porteus:~/di#
root@porteus:~/di/stockfish-dd-src-src-1/src# make help

To compile stockfish, type: 

make target ARCH=arch [COMP=comp]

Supported targets:

build                   > Standard build
signature-build         > Standard build with embedded signature
profile-build           > PGO build
signature-profile-build > PGO build with embedded signature
strip                   > Strip executable
install                 > Install executable
clean                   > Clean up

Supported archs:

x86-64                  > x86 64-bit
x86-64-modern           > x86 64-bit with popcnt support
x86-32                  > x86 32-bit with SSE support
x86-32-old              > x86 32-bit fall back for old hardware
linux-ppc-64            > PPC-Linux 64 bit
osx-ppc-64              > PPC-Mac OS X 64 bit
osx-ppc-32              > PPC-Mac OS X 32 bit
osx-x86-64              > x86-Mac OS X 64 bit
osx-x86-32              > x86-Mac OS X 32 bit
armv7                   > ARMv7 32 bit
general-64              > unspecified 64-bit
general-32              > unspecified 32-bit

Supported compilers:

gcc                     > Gnu compiler (default)
mingw                   > Gnu compiler with MinGW under Windows
clang                   > LLVM Clang compiler
icc                     > Intel compiler

Non-standard targets:

make hpux               >  Compile for HP-UX. Compiler = aCC

Examples. If you don't know what to do, you likely want to run: 

make build ARCH=x86-64    (This is for 64-bit systems)
make build ARCH=x86-32    (This is for 32-bit systems)

root@porteus:~/di/stockfish-dd-src-src-1/src#
How do I have to change the instructions of src2pkg so to get the compiling of the source package straight?
It shoud be something like:
make build ARCH=x86-32 [COMP=gcc] DESTDIR=/root/di/
Thanks.
Voltaire: Le mieux est l'ennemi du bien.

Post Reply