tip - shrinking down your own modules

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

tip - shrinking down your own modules

Post#1 by Rava » 26 Jul 2013, 11:05

Hi folks,

I thought I share with you my attempts at shrinking down my own modules, especially for the use on systems with low RAM and / or old CPU...

A warning: editing / deleting files from a module can break something, so be sure only to remove files from doc / man / locales folder, no binaries, as in /usr/bin or /bin or /sbin or such, also no libraries, as in /usr/lib or /usr/lib64 or /lib or /lib64!

Stripping especially complex programs, such as GIMP or inkscape of unneeded locales to save file space, and especially uncompressed space.

In my below example, we want to create a stripped down GIMP module, that includes GIMP's mandatory babl and gegl libraries.

First, you need to download the modules, in this case, I downloaded gimp-2.6.11-x86_64-3.xzm, babl-0.1.2-x86_64-1.xzm and gegl-0.1.2-x86_64-1.xzm from ponce.cc and saved them in the /tmp directory. You can save it in any directory you want, but do use a linux filesystem, when creating the folder for the new module on a vfat or NTFS FS, the file attributes and ownerships of files and folders will be messed up, so, either put it in your live system, e.g. on /tmp, or, when you not have enough RAM to do so, create the folder in a linux ext2/ext3/reiser/whatever-linux-FS of your choice.

(Of course, knowing that only babl and gegl is mandatory for GIMP, and that the other modules PPM suggests are not mandatory, but expendable takes experience... since I am a fan of optimising my modules, I tried it for some years to keep my modules as small as it can be, including only adding the really needed dependencies.)

Second, we create the new directory, best naming it like we want to name the finished module:

Code: Select all

root@porteus:/tmp# mkdir gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped

Then we need to extract the modules into that directory
:

Code: Select all

root@porteus:/tmp# xzm2dir gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped
[ ... output of xzm2dir snipped ... ]
root@porteus:/tmp# xzm2dir gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped
[ ... output of xzm2dir snipped ... ]
root@porteus:/tmp# xzm2dir gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped
[ ... output of xzm2dir snipped ... ]
Then I removed all folders and files in that folders that have locales for the program(s) that I don't need, that is, all but de (German) and en-UK (British English).
The locales you always find in usr/share/locale, after deleting my folder looks like this:

Code: Select all

root@porteus:/tmp/gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped/usr/share/locale# ls -o
total 0
drwxr-xr-x 3 root 60 Mar 22  2011 de
drwxr-xr-x 3 root 60 Mar 22  2011 en_GB
Of course, here you keep the language or languages you want to use, and delete all others. When you are okay with the standard, that is: US English, you can even delete all locale folders.

Then, I do another trick, but that will break the rules for GNU, BSD and other licences, so only do it when you NOT share the modules with others.
Since I deem it unnecessary that I have lots of identical COPYING files on my system, that have lots of redundancy, I create an empty file that holds the name and version of the actual licence file in its filename, and remove the actual licence file.
When someone want to read the licence file, (s)he just have to look it up, most probably on the very system (s)he is running, since it will be there somewhere already.

Code: Select all

root@porteus:/tmp/gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped# ls -o usr/doc/*/COPY*
-rw-r--r-- 1 root    0 Jul 24 10:00 usr/doc/babl-0.1.2/COPYING.GNU LESSER GENERAL PUBLIC LICENSE V3
-rw-r--r-- 1 root    0 Jul 24 10:00 usr/doc/gegl-0.1.2/COPYING.GNU GENERAL PUBLIC LICENSE V3
-rw-r--r-- 1 root    0 Jul 24 10:00 usr/doc/gegl-0.1.2/COPYING.LESSER.GNU LESSER GENERAL PUBLIC LICENSE V3
-rw-r--r-- 1 root    0 Jul 24 10:00 usr/doc/gimp-2.6.11/COPYING.GNU GENERAL PUBLIC LICENSE V2


Now create the new module with

Code: Select all

root@porteus:/tmp# dir2xzm gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped/ gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped.xzm
Let's see how much space we saved:

Code: Select all

root@porteus:/tmp# ls -o *xzm
-rw-r--r-- 1 root    65536 Jul 24 09:56 babl-0.1.2-x86_64-1.xzm
-rw-r--r-- 1 root   372736 Jul 24 09:56 gegl-0.1.2-x86_64-1.xzm
-rw-r--r-- 1 root  7348224 Jul 24 10:04 gimp-2.6.11-x86_64-3+babl+gegl_locales-stripped.xzm
-rw-r--r-- 1 root 14381056 Jul 24 09:56 gimp-2.6.11-x86_64-3.xzm
See?
7348224 bytes for the whole GIMP including babl and gegl, instead of (added up) 14819328 bytes for all 3 original modules...
(Or in Megabytes: 7.01 MB instead of 14.13 MB)

I hope this is helpful to some of you, but please be aware: if you want to share your modules, better keep the COPYING* files as they are...
Cheers!
Yours Rava

donald
Full of knowledge
Full of knowledge
Posts: 2064
Joined: 17 Jun 2013, 13:17
Distribution: Porteus 3.2.2 XFCE 32bit
Location: Germany

Re: tip - shrinking down your own modules

Post#2 by donald » 26 Jul 2013, 11:18

@Rava

That's cool man.... 8)

User avatar
francois
Contributor
Contributor
Posts: 6434
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

Re: tip - shrinking down your own modules

Post#3 by francois » 16 Oct 2016, 11:36

Nice tip!

But if you make the module for the porteus community, what could be stripped? :)
Prendre son temps, profiter de celui qui passe.

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Re: tip - shrinking down your own modules

Post#4 by Rava » 15 Nov 2016, 06:49

francois wrote:But if you make the module for the porteus community, what could be stripped? :)
For the community as a whole, well, not much.

We could agree on having the 5 or so mainly used licence files, like GNU or such only once in the system, and all other packages have just a softlink to that one file already in the system. Since a Porteus module is not meant to be used stand-alone it should be okay with the way these licences are meant to be.

They are not that large, but when you have dozens of dozens, it gets large...


But back to the individual packages... You cannot really strip it when creating modules for the community, unless you want to show off why the thread is named "shrinking down your own modules", since you know which one or no more than two languages you want to keep, and remove all of the rest and then upload both versions, and name one e.g. name-1.2.4_STRIPPED_en_UK.xzm and explain that this "name" only comes in en_UK and with no other language, and serves as tip how much size one can save when adjusting his/her own module...
Cheers!
Yours Rava

Post Reply