Porteus-AUR: Building Porteus modules w/ Arch build scripts!

[roelof]

Python fails with this message :

[371/371/2] test_zlib
340 tests OK.
2 tests failed:
test_strptime test_time
2 tests altered the execution environment:
test_site test_urllib2_localnet
27 tests skipped:
test_codecmaps_cn test_codecmaps_hk test_codecmaps_jp
test_codecmaps_kr test_codecmaps_tw test_curses test_dbm_ndbm
test_devpoll test_gdb test_idle test_kqueue test_msilib
test_ossaudiodev test

Roelof

[brokenman]

@roelof
Somewhere in the build script you will need to comment out the chown line since there is no group called systemd-journal in porteus. If this breaks anything you will need to add the group (groupadd) before trying to build.

As root you can type: groups to see what groups exist. The other errors are minor and caused because porteus strips most documentation.

@ViktorNova
don't care what version of a lib you have, as long as the lib/headers are found.
That's a recipe for disaster.

There is not much more I can offer you at the moment. The plan is to have USM replace PPM so that nobody has to maintain a porteus repository. No need to double handle packages this way. PPM has many requirements when building a package and infact won't deal with any packages that it didn't create/download itself.

I would need to see the database file for the repository to offer anything useful. The process (for dependency resolution) will be:

1) Download pkg
2) build pkg
3) Run ldd against all libraries and executables in the package and see what is "not found"
4) Search slackware for the mother package of these libraries (and download)
5) If not found in slackware then search slackbuilds, then arch
6) Download these dependencies and repeat this loop until everything is resolved.

This is why precompiled packages are very useful. The difficult part will be during the build. If the SLKBLD fails because it missing a build time dependency then automating anything will be difficult if not impossible. Human intervention will be needed to read the error, make a decision on what is needed and then act. Same thing applies for slackbuilds.

[roelof]

Thanks,

I will try that.
The python error can be ignored because it python3 and I need python2.

Roelof

[francois]

Under porteus kde 2.1, building nano was sweet.

In the process of building gnumerics, it was not so easy as it required more recent version of certain packages or libraries<; gtk+3-3.4.4 and libgsf--1.14.28-1, goffice-0.10, gobject-introspection-1.32.1... that could be fetch from porteus, slackware, slacky and pkgs.org. Additional, libraries had to be built from Arch: libgsf-1.14.28.

I am not done yet, but the process of building packages from arch structure is rather pleasant.

Thanks.

[francois]

multilib is downloading now.

[phhpro]

...

[francois]

We will have to see if AUR packages could be seemlessly incorporated to usm package manager. 8)

[ViktorNova]

@francois
Glad you dig it! It's neato, huh? 8) In my experience, even having to manually track down all the dependencies and install many of them from Arch, it's been much easier for me to get the latest version of lots of weird software running issue-free on Porteus this way than going for the pure source or Slackbuilds (although every once in awhile the Slackbuild will succeed where this will fail, always because of a certain patch and/or install script)

@phhpro
The reason this only works with -d right now using this particular method, is that the only way Arch's makepkg would knows if a dependency exists is to check if that dependency has been installed by Pacman, Arch's package manager. So basically, it instantly fails and says you're missing dependencies, even if you actually do have all the dependencies satisfied already. Hopefully, this will change in the future =D

@brokenman
Thank you, this totally helps me on this mission. I have a few questions - by database, do specifically mean sql? And would this db also need to include runtime/build dependencies? This may well exist already, I'm just not a hardcore enough Arch user to know off the top of my head, but if this is what you'd need for USM I will totally do some digging. The commandline ABS/AUR search tools I have used and am currently looking at tweaking are returning results in json (package names, versions, and descriptions) and then when a package is selected, the PKGBUILD bash script is downloaded and parsed for dependencies.

I actually have not looked at USM, do you have a way to deal with dependencies of Slackbuilds at the moment, or are you not going for Slackbuild compatibility? Does USM track dependencies with ldd alone, or are there other things it checks too?

@roleof
With something as deep in the core of the OS as systemd, you can expect to have some difficulty any way you approach it, you are ambitious! Arch's way of installing it for Arch is likely to have some non-slackware/porteus compatible things there.. maybe not though. I happened to notice this on Github today and thought I'd pass it on, might be less problematic?
https://github.com/PhantomX/slackbuilds ... er/systemd

[ViktorNova]

I thought I'd mention this in case it helps someone: sometimes when building packages from Arch, a lib that exists on your system might occasionally not get picked up during the compile process. I've done a few things to help this situation:

-running ldconfig as root.
Don't know why, but every once in awhile this picks up a bunch of new things

-Adding this to your users ~/.bashrc
PKG_CONFIG_PATH=/usr/lib/pkgconfig:/usr/local/lib/pkgconfig:/usr/lib64/pkgconfig

-adding /usr/lib to /etc/ld.so.conf
(if it's not already there) and then running ldconfig as root

[brokenman]

by database, do specifically mean sql?
No. Whatever file holds the information about the AUR packages. In slackware repositories this is plain text.

The commandline ABS/AUR search tools I have used and am currently looking at tweaking are returning results in json
Whatever the CLI search tools are parsing for information will be the database file. That handles the package information, great. Then if all is needed is the PKGBUILD to get the dependencies (and they are accurate) then you have what you need to make it work.

do you have a way to deal with dependencies of Slackbuilds at the moment
In slackware it is not really required because a full slackware install contains everything you need to build your packages (same with dependencies). Nothing should fail. Porteus is minimal so many packages are missing (QT in some desktops for example) and if a SlackBuild requires QT in this situation it will fail. I am slowly working towards a system for discovering and automating the slackbuilds process for Porteus but as I said above it is not easy to do this for package building. Think of all the times you compiled something and it failed. The current release of USM does not handle dependency resolution for SBo. I don't a tool exists yet for this.

Does USM track dependencies with ldd alone, or are there other things it checks too?
There are two files that USM requires to handle dep resolution. One contains a list of all files a package contains (MANIFEST) and the other contains a list of all libraries a package requires to run (LIBS.TXT). USM itself doesn't run ldd except in one situation, when a user wants to resolve dependencies for a package they downloaded manually. I have built in a file so that if a dependency is required but missed by USM (for whatever reason) then it can be manually added. This was the case for clucene and audacious.

Hope that helps. Good luck.

[francois]

@ViktorNova:

@phhpro
The reason this only works with -d right now using this particular method, is that the only way Arch's makepkg would knows if a dependency exists is to check if that dependency has been installed by Pacman, Arch's package manager. So basically, it instantly fails and says you're missing dependencies, even if you actually do have all the dependencies satisfied already. Hopefully, this will change in the future =D

Do you mean that you would fool pacman or abs and feed them with a list of all the packages and libraries of porteus stock?

[tome]

Are there any database files that would allow a program to search for and report available packages from within the program? It would be nice to do all searching and downloading without having to switch to a browser.

Some trick is to create a list with all directories in /var/abs, convert to proper text file and after invoke extract pkgver= from /var/abs/.../PKGBUILD

[ViktorNova]

@tome
Good idea! I'll do that. I've found a thing (not on github yet) that downloads the entire list of all Arch & AUR package name, versions, and descriptions, and outputs to stdout, so that will be easy

@phhpro
Thats the idea! More realistically, it will probably be a hack that uses Porteus/Slack tools to check if listed dependencies are installed, optionally install them, and then continue

@brokenman
Thanks for all this info, I got some work to do! I will report back when I've made some headway

@roelof
Just this week I compiled a ton of Python stuff and learned a few things-Arch used Python 3 by default, and on Arch "python" means Python 3, and "python2" is the Python we're more used to. In order to get stuff to work on Porteus, I first had to do a few things:

First, I installed python3 from SBo (as I compiled some audio tools that use Python 3)
http://slackbuilds.org/repository/14.1/python/python3/

For most Python packages, the PKGBUILD has 2 (or 3) sections - one for python2, one for python3, and sometimes one for common shared stuff between the two

After that, most python-related PKGBUILDs needed the following changes:

1: Replace all references to /usr/lib/ with /usr/lib64/
2: Replace all commands that start with "python" with "python3"

After that, python stuff should work better!

[francois]

Just to have some water flow in the threadmill. Hoping that the following comments will not be too much out of track.

do you have a way to deal with dependencies of Slackbuilds at the moment
In slackware it is not really required because a full slackware install contains everything you need to build your packages (same with dependencies). Nothing should fail. Porteus is minimal so many packages are missing (QT in some desktops for example) and if a SlackBuild requires QT in this situation it will fail. I am slowly working towards a system for discovering and automating the slackbuilds process for Porteus but as I said above it is not easy to do this for package building. Think of all the times you compiled something and it failed. The current release of USM does not handle dependency resolution for SBo. I don't a tool exists yet for this.

@brokenman:
I have done some research on the following theme to found interesting tools (for you to judge):
I don't (think) a tool exists yet for dependency resolution.

spbuilder for slackware: generates a new, completely separate, minimal Slackware environment in which the package's build script is executed.
http://www.vislab.uq.edu.au/howto/spbui ... ilder.html

scripts to collect and visualize dependency graphs from SlackBuilds.org
http://slackalaxy.wordpress.com/tag/sbo/

sbopkg and sbotoos coexistance:
https://slackalaxy.wordpress.com/2012/1 ... s-coexist/

sbopkg ... ... combined with a collection of queue files (sbotools), the dependency resolution becomes semi-automatic.

sbotools, a recent project:
http://dawnrazor.net/sbotools/

[amigo]

The trouble with all the sbo deps info is that they assume a complete installation as the base. So even the info they provide is partial and helps not one bit when trying to cut down to less than a complete installation. AUR PKGBUILD scripts also are not ideal as they contain arbitrary depends info which must be manually generated.