[IMPORTANT] Package manager for next release

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
User avatar
brokenman
Site Admin
Site Admin
Posts: 5461
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v3.2rcX all desktops
Location: Brazil
Contact:

[IMPORTANT] Package manager for next release

Post#1 by brokenman » 31 Dec 2010, 21:42

One of the beautiful things about most linux flavours is the ability to search and install packages with a simple click or command. At the moment our UknownX distro uses the lzm_ fiels to accomplish this (for one method). i really like this but i feel the system could be improved and we could start using our own official server.

At the moment i am getting a server error for lzm_list. Falcony has done a great job with this and it is very inspirational. I would propose the following for a system to work from our official server:

A simple text file sits on the server similar to packagelist.txt in slackware. This file contains a list of packages available, a description and even a list of files contained within the module. When someone queries a package this file is downloaded and searched. Creation of this file can be automated once a week by way of a serverside script that loop mounts every module, sniffs it and dumps the info to packages.txt. The days date would be the first line yymmdd for update checks.

I feel this method would speed up queries, and does not rely on complex lynx functions to grab lists and descriptions. If anyone would like to undertake this project that would be great. If not i can pursue it further.

Please feel free to agree/disagree or post your opinions on this!

BTW: Big shout out to ponce for supplying such an awesome (and vital) part of the system. The server space! This is all pending ponce's approval of running a serverside script.
How do i become super user?
Wear your underpants on the outside and put on a cape.

User avatar
X-Factor
Black ninja
Black ninja
Posts: 88
Joined: 29 Dec 2010, 23:19
Location: bali, indonesia

Re: Package manager

Post#2 by X-Factor » 31 Dec 2010, 23:43

:good:
I've been confine to my quarter
http://img143.imageshack.us/i/medusathing.jpg/

User avatar
fanthom
Site Admin
Site Admin
Posts: 4566
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland, currently - Cork, IE
Contact:

Re: Package manager

Post#3 by fanthom » 01 Jan 2011, 05:10

keeping packages up to date, looking after conflicting libraries version means a lot, a lot and a lot of work. i have a bad experience from slax.org. i do understand that real distro must have a good repo but i dont know if we are ready for this now.

Anyway, good that this idea is pointed - we can try as we have nothing to loose :)
Please add [Solved] to your thread title if the solution was found.

User avatar
snake
White ninja
White ninja
Posts: 14
Joined: 29 Dec 2010, 10:02
Distribution: Porteus-v3.1 64bit KDE
Location: Finland

Re: Package manager

Post#4 by snake » 01 Jan 2011, 19:11

I really liked the feature on slax.org modules that you could "active" program you want to use from web page, using "slix://..." url. We should have that feature on package repository. There are a programs that I need to use once in month or less, for example Inkscape, and it would be great to just click slix-like link on web page to start using the software without installing it. Or it could be even included on application menu. Problem with slax repository is that users can upload packages, so you cannot trust the content, and there are lots of outdated versions, and depending libraries are missing. Fixing this, adding automatic checksum feature, and generating packages automatically is a job todo. and generating/including text file with output of unsquashfs -ls package.lzm.

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: Package manager

Post#5 by francois » 02 Jan 2011, 14:09

I like snake proposition. Packages in form of modules should come with all necessary libraries.

Isn't there some way of using optimally the work that has been already built by others. After all slackware is a distribution that already works per se. Other distributions based on slackware function with their own package manager like zenwalk or zenlive, or a general package manager used under slackware slapt-get, salix works with slapt-get. Slapt-get could be the general primary front-end to manage packages.

There is even a module made by Some-guy based on slapt-get called slax-get, it would have been updated from slackware 12.0 to work with slackware 13.0 by dssmex:
http://www.slax.org/forum.php?action=vi ... ostid72277
http://www.slax.org/modules.php?action=detail&id=3897

Gslapt is a real nice frontend for slapt-get. Quite alike synaptic for apt-get and the Debian packages.

For those looking for more lightweight package solutions, maybe we could find a way to validate slax modules with some kind of a tester criteria. For example, given that half a dozen of remix registered user found a given module functional on their boxes, the module could be recognized as being validated. Some comments associated with each module could be made by users. This way in the case of some exceptional difficulties with the given module it could be discussion in association with the module reffered to.

Maybe a little frontend permitting the integration of the given module with all the necessary libraries into an overall out-of the box module could be of some use.

By the way how is your experience with remix Slax module center yet? For one reason or another, I am not able to use the package manager. Detailed howto would be appreciated, if it is not already done.


*******************************************************
Edit 2011-01-02.

Concerning modules and package managers:

For the basis of this discussion, it seems a very good thing read Slax-remix-v09 32-bit FAQ, items 8, 18, 19 and 21:
viewtopic.php?f=42&t=35

An old thread that I innocently began a while ago centered on the issue of package management, it is really not bad to have a look at it, some power users comment the issue:
http://www.slax.org/forum.php?action=vi ... t=some-guy

Searching, reading and finding scarce information on package management in the remix threads from ver 05 to 09, I got some information on slapt-get and gslapt (was not an option for fanthom), slakyd (integrated in the os, but a dead end as the software will no more be maintained, but still useful now to resolve dependencies, a must see remix facts item 18 hyperlink above), on the slax module center (promising software, but finally non-functional), and on slackpkg (Quax would be working on that one, to be included eventually in the OS).

It seems that a brief and more informed report by some of the most concerned by package management in Remix, so that we get a better summary of the situation would be warmly welcome.
Thanks.
Voltaire: Le mieux est l'ennemi du bien.

Falcony
Full of knowledge
Full of knowledge
Posts: 233
Joined: 01 Jan 2011, 12:44
Location: Russia

Re: Package manager

Post#6 by Falcony » 04 Jan 2011, 09:59

Dear all,

I think it have no sense to use centralized database of depedices which download and store at Live FS

This is solution hard and required special people - team who responsible only for creation of modules and repositories adding, deleting and maintaining package databese.

This is VERY TIME consuming matter - and I see we here do not have enough resources to maintain such complex thing.

My prposition

Let each package creator include nessesary information inside each lzm module.

This solution do not reqired centralized repository management but provide reliable informaition to "cloud" package manager.

Take in the mind these consideration:

1. name of module
2. description of modules
3. version of module
4. creator information - name or nick and e-mail address
5. list of depends
6. links to repositories where depends avalaible
7. minimal version for each depends
8. list of conflicts
9. list of included modules(as some of creator of modules like to include to module several programs - but package manager have to know about installled software)

So client tool for managing modules in my consideration have to be simple -

1. Download module
2. Activate module
3. After activation see information for depends, links to repositories and conflicts
4. Proceed download nessairy depends or ask user to deactivate some modules(if confilct)

We need our own standard - which module creators have to follow. And this standard of lzm modules should have no any relation to Slackware

Or else we will have similar situation as Tomas M have with a users modules - good and best Live distro itself and bad and even worse and sometime unworkable user's modules

My implementation of FIDOSlax package management please find at these topic

http://porteus.org/forum/viewtopic.php? ... p=436#p436

User avatar
fanthom
Site Admin
Site Admin
Posts: 4566
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland, currently - Cork, IE
Contact:

Re: Package manager

Post#7 by fanthom » 04 Jan 2011, 15:29

Falcony wrote:We need our own standard - which module creators have to follow. And this standard of lzm modules should have no any relation to Slackware
hmmm... this is rather revolutionary idea :)
i think we should follow slackware standards as we are using slackware as a base. Even when some packages comes from other distros (like gentoo, debian, fedora), they should be 'rebranded' on slackware format to be easy manageable by slackware tools: pkgtool, slackyd, etc.
I think you could create 2 standards for your packages: FIDOSlax and Slackware.
Here is an idea of a wrapper for you:
a) unpack existing FIDOSlax lzm
b) enter unpacked dir and create /install/slack-desc file with short description (syntax of this file is little bit tricky - have a look on a sample 'txz')
c) strip package: decompress man pages, strip libraries from debug symbols
d) create standard slackware txz package with 'makepkg' utility:
makepkg -l y -c n ../$pkgname-$arch-$build.txz (providing proper pkgname, arch and build variables is very important)
e) convert txz to lzm
f) voila!
Packages created with this way could be accepted in our repo (i think but it may change). Other packages will have to live in external repos as unofficial/unsupported.
Please consider my suggestion.
Please add [Solved] to your thread title if the solution was found.

Falcony
Full of knowledge
Full of knowledge
Posts: 233
Joined: 01 Jan 2011, 12:44
Location: Russia

Re: Package manager

Post#8 by Falcony » 04 Jan 2011, 21:16

fantom, I know what Slackware packages is and how to create them

But

Please kindly divide Slackware packages tgz/txz and Slax lzm modules

I see here differnence:

=====
Slackware packages stored at HDD - so limitation of space is not matter basically
Slax modules loads to RAM - limited to size of RAM

Number of installed Slackware packages could be bigest, not unlimited but quite big
Number Slax modules at once loads to RAM very limited and limeted to several hungred modules

Slackware packages are static as soon as installed all depend stick to libraries and binaries for long time and periods
Slax modules are dynamic - clould be loaded/unloaded on the fly during short period of time

Slackware has sets of packages
Slax in slax/base has bunlded of system modules and user modules in slax/modules

Slackware system has much biggest baseline - very stable and the whole system itself on HDD provided by Patric takes 5-7 Gb
Slax system has lowers baseline - very sable and the whole system itself is about 200-250 Mb

Slackware system is very stable and the whole system itself on HDD provided by Patric takes 5-7 Gb
Slax system has lowers baseline - very stable and the whole system itself is about 200-250 Mb

Slackware packages beyond of baseline - created by user itself by Slackbuilds and waste own time or use forein repos with much worse quality of packages - and got very unstable system
Slax system beyond baseline - create by user itself and waste own time or use other people modules and got finally very unstable system

Slackware packages usually have only one program or library in a package
Slax module could be bundle and contain number of Slackware packages - so if you got to such bundle from different
======

Please look in face of facts - Slackware packages and Slax modules are different kinds and used another ways

Tomas M also do not divide Slackware packages and Slax modules

So as Slax users - who created lzm modules other ways - and mostly of this modules was in bad quality

There are many module creators who are not follow the rules as there no right rules for packaes

Some of them creates modules converting Slackware packages or other distro
Some of them use SlackBuilds
Some of them put number programs and libraries to one module

And main problem

Slackware packages do not contains information regarging depends and conlicts
So as Slax modules

As these reasons many and dirty modules are in bad condition

I think Tomas M try to manage this - I see information in his blog that he found some guy who checked modules created by users - but without success

Tomas M package managements system via Web-site - "Build your own Slax" failed by uppers reasons

Whitout clear standard of lzm module and package manager which cloud handle depens and conflict soon we repeat current Slax situation

That is no good for users and for disro

Please think of future - what happen if you distro become such popular as Slax?

The same mess with user's builded packages as Tomas M have!

Why we need to repeat the same Tomas M mistakes?

He is great guy but he think like a boy - without seeing perspective
Slax just a toy for him, toy for playing

So my proposition:

1. There should be strict rules which each module creator have to follow
2. Each module should have package information inside of each module - may be even links for getting depends - or there should be special team who will be resposble for maintaining offical repository and checking packages and update list of modules and depends
3. There should be package manager which automatically download and activate modules, stop at confict, download depends
4. If lzm module do not have proper package information iside - it should be blocked and not activated by activate script

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: Package manager

Post#9 by francois » 05 Jan 2011, 01:21

fanthom cited:
Even when some packages comes from other distros (like gentoo, debian, fedora), they should be 'rebranded' on slackware format to be easy manageable by slackware tools: pkgtool, slackyd, etc.

****************************
And what about pkgsrc (package source) the package management system for Unix-like operating systems:
http://kuparinen.org/martti/comp/slackw ... kware.html
http://www.netbsd.org/gallery/10years.html#alcrooks

Download:
http://www.netbsd.org/docs/software/pac ... #platforms
Voltaire: Le mieux est l'ennemi du bien.

Falcony
Full of knowledge
Full of knowledge
Posts: 233
Joined: 01 Jan 2011, 12:44
Location: Russia

Re: Package manager

Post#10 by Falcony » 05 Jan 2011, 09:07

I like snake proposition. Packages in form of modules should come with all necessary libraries.
May be, may be - discussion is required. And idf select suck behave all creator have to use this rule as common rule and have to be in a policy and list of activated libraries and program have to be avalible to package manager
There is even a module made by Some-guy based on slapt-get called slax-get, it would have been updated from slackware 12.0 to work with slackware 13.0 by dssmex:
I my opition Slap-get or other tool may be useful for getting and converting packages. But it is for Slackare. As I said upper Slackware package and Slax modules are different kinds and using different way so plain conversion will bring mess to a system and waste user time
And what about pkgsrc (package source) the package management system for Unix-like operating systems
I used pkgsrc at Linux(not live) and may say - pkgsrc works badly
When you need build small utlities and program it is OK

But when you need to build some X application - it tried to build own X and etc.

So use FIDOSlax build tools - arch-build, arch-dep, crux-build and slax-build -
it creates lzm modules from source using rules for building of SlackBuilds, Arch Linux ABS and CRUX Linux ports and do not require rebuild Xorg or other huge application

User avatar
fanthom
Site Admin
Site Admin
Posts: 4566
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland, currently - Cork, IE
Contact:

Re: Package manager

Post#11 by fanthom » 05 Jan 2011, 10:14

@Falcony
great insights! I think that everyone should read your posts in this topic. It may really be an improvement to add 'dependencies' section to (for example) /var/log/packages/* file.
I see one downside of building packages from sources (i have 5 years experience with gentoo) by everyone:
-they will fit in 100% only system on which they were build. Let's say that user is building 'mplayer' and has no opencore-amr package in his system. in case of sharing mplayer.lzm, other user which has opencore-amr wont be able to use this codec.
Slackware packages are built by professionals, even if normal user converts them to lzm and share - they are still a high quality packages.
Method suggested by you is good but we can finish like Arch at the moment:
5 versions of kernel in AUR, each built with different set of features.
(i can imagine 10 versions of mplayer: one build with irda support, second one with vdapu, etc...)

This subject must be discussed extensively - made sticky (and changed topic a little)
Please add [Solved] to your thread title if the solution was found.

Falcony
Full of knowledge
Full of knowledge
Posts: 233
Joined: 01 Jan 2011, 12:44
Location: Russia

Re: [IMPORTANT] Package manager for next release

Post#12 by Falcony » 05 Jan 2011, 12:38

It may really be an improvement to add 'dependencies' section to (for example) /var/log/packages/* file.
Protes module could have (and for big modules like base - always it is) several number Slackware packages inside

So better if we competely separate Proteus modules and Slackware packages iinformation

Why do not have special directory for example

/var/proteus

to store Proteus modules information?

And in the same time /var/log/packages/* will be as it is now


So for example if we have application installed in a system then we have module information records in

/var/proteus/directory

number some files files for ex. :

/var/proteus/directory/CONFICTS - with what lzm modules it conflicts
/var/proteus/directory/DEPENDS - what modules it required
/var/proteus/directory/LINKS - list of repos/mirrors depens could be downloaded
/var/proteus/directory/DESC - desciption of module
/var/proteus/directory/INCLUDE - from what what slackware packages it is consists(the lists)
/var/proteus/directory/MAINTAIN - Maintainer info
...etc

For ex. for AbiWord we will have


/var/proteus/abiword/DEPENDS:

000-kernel.sq4.lzm
001-core.sq4.lzm
002-xorg.sq4.lzm


/var/proteus/abiword/LINKS:

http://proteus/repo1
ftp://proteus/repo2

etc..

/var/proteus/abiword/DESC:


The AbiWord processor

/var/proteus/abiword/INCLUDE:


librsvg-2.32.1-SBo.tgz
gnome-vfs-2.24.3-SBo.tgz
libbonobo-2.24.3-SBo.tgz
gnome-mime-data-2.18.0-1--SBo.tgz
goffice-0.8.10-SBo.tgz
gconf-2.24.0-1-SBo.tgz
libidl-0.8.14-1-SBo.tgz
libgnomecups-0.2.3-1-SBo.tgz
fribidi-0.19.2-1-SBo.tgz
gnutls-2.10.4-SBo.tgz
orbit2-2.14.19-SBo.tgz
wv-1.2.7-SBo.tgz
libtasn1-2.8-SBo.tgz


/var/proteus/abiword/MAINTAIN:

Falcony, mfalcony at rambler.ru

Method suggested by you is good but we can finish like Arch at the moment:
5 versions of kernel in AUR, each built with different set of features.
(i can imagine 10 versions of mplayer: one build with irda support, second one with vdapu, etc...)
I ubderstand you concern

My opinion - better block module activation if module created to use previos versions its depends.

For example mplayer which sticked was builded with some option to kernel version X should have depends information

/var/proteus/mplayer/DEPENDS:

000-kernel.versionX.lzm
001-core.versionX.lzm
002-xorg.versionX.lzm


In case of future Protus system update in slax/base will be other proteus modules

000-kernel.versionY.lzm
001-core.versionY.lzm
002-xorg.versionY.lzm


Why user should have in a system semi-workable version of software or even completely unworkable?

So in this case activation scripts should prevent activation mplayer module

EDIT:\\
proper name for our project is 'Porteus' and not 'Proteus'
fanthom

User avatar
brokenman
Site Admin
Site Admin
Posts: 5461
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v3.2rcX all desktops
Location: Brazil
Contact:

Re: [IMPORTANT] Package manager for next release

Post#13 by brokenman » 05 Jan 2011, 20:45

i like this idea. It is what i originally suggested early on in the show. An index file within every module that contains list of dependencies. Very good.
How do i become super user?
Wear your underpants on the outside and put on a cape.

Falcony
Full of knowledge
Full of knowledge
Posts: 233
Joined: 01 Jan 2011, 12:44
Location: Russia

Re: [IMPORTANT] Package manager for next release

Post#14 by Falcony » 05 Jan 2011, 22:08

@fantom,

ok. And Porteus - sounds better L)

@brokenman,

Another better and most known solution - to have official repo and team - several number stuff who check modules add it to repo and update database of packages and depends.

All clients pc run one command for updationg database
and several other commands to manage local packages

Many pluses - rock stable

And one minus - it required staff who work with "durty" work and not interest work

User avatar
X-Factor
Black ninja
Black ninja
Posts: 88
Joined: 29 Dec 2010, 23:19
Location: bali, indonesia

Re: [IMPORTANT] Package manager for next release

Post#15 by X-Factor » 06 Jan 2011, 00:27

8), count me out though as my resource are VERY limited :cry:
I've been confine to my quarter
http://img143.imageshack.us/i/medusathing.jpg/

Locked