How do you make Porteus so small?
-
- White ninja
- Posts: 13
- Joined: 10 Jan 2012, 00:10
- Location: The Independent Republic of Massachusetts
How do you make Porteus so small?
Recently I experimented with making a custom live CD using Remastersys on a Debian system with Xfce. Imagine my surprise when the ISO came out to over 560 MB!
This is especially weird because:
- Debian + Xfce should really have fewer libraries floating around than Slackware + Trinity
- Debian has dependency resolution, so only what's needed gets installed
- Debian puts headers in separate packages (which I didn't install)
- My Debian installation basically had all the same system stuff as Porteus: CUPS, wicd, etc.
And yet... the ISO image is still huge. What did you guys do to get an image smaller than 250 MB? Is LZMA compression that good?
This is especially weird because:
- Debian + Xfce should really have fewer libraries floating around than Slackware + Trinity
- Debian has dependency resolution, so only what's needed gets installed
- Debian puts headers in separate packages (which I didn't install)
- My Debian installation basically had all the same system stuff as Porteus: CUPS, wicd, etc.
And yet... the ISO image is still huge. What did you guys do to get an image smaller than 250 MB? Is LZMA compression that good?
- Ahau
- King of Docs
- Posts: 1331
- Joined: 28 Dec 2010, 15:18
- Distribution: LXDE & Xfce 32/64-bit
- Location: USA
Re: How do you make Porteus so small?
It's a combination of factors, and a lot of effort and thought put into keeping the size down. LZMA2+squashfs is pretty awesome for compression, so that helps. Dependency resolution in your package manager will make your ISO bigger, not smaller, because if you are manually putting your packages together you can choose to leave out some dependencies (i.e. leave them unresolved, and there are a couple in Porteus like this) or compile them out. Once you learn the ins and outs, you can put together a lean system without automatic dep resolution. There's also a bunch of files included in many packages that aren't needed for running the system, and a lot of those are stripped out of porteus.
When I started working with XFCE 4.8, I think the whole module was around 35MB, but it's down to just under 10MB in 1.2 RC2.
When I started working with XFCE 4.8, I think the whole module was around 35MB, but it's down to just under 10MB in 1.2 RC2.
Please take a look at our online documentation, here. Suggestions are welcome!
-
- White ninja
- Posts: 13
- Joined: 10 Jan 2012, 00:10
- Location: The Independent Republic of Massachusetts
Re: How do you make Porteus so small?
Wow, that is quite impressive. I think Thunar alone drags in more than that on Debian!
Out of curiosity, can you provide any specifics on what sort of stuff gets stripped out? e.g. what would you leave out of, say, the Firefox module?
Out of curiosity, can you provide any specifics on what sort of stuff gets stripped out? e.g. what would you leave out of, say, the Firefox module?
- Ahau
- King of Docs
- Posts: 1331
- Joined: 28 Dec 2010, 15:18
- Distribution: LXDE & Xfce 32/64-bit
- Location: USA
Re: How do you make Porteus so small?
Using xfce as an example, I cut out a number of unnecessary dependencies (such as gconf, ORBit, gnome-keyring, guile, glade3 and everything gstreamer related), all locales files (/usr/share/locale/*) and localized man pages get split off into a locales module (for use with Porteus Language-Selection-Tool), and I drop everything from /usr/info and /usr/doc.
Firefox isn't as good of an example because there are fewer packages and less to cut out (but localization and documentation might be a good place to start if you don't need them).
Firefox isn't as good of an example because there are fewer packages and less to cut out (but localization and documentation might be a good place to start if you don't need them).
Please take a look at our online documentation, here. Suggestions are welcome!
-
- White ninja
- Posts: 13
- Joined: 10 Jan 2012, 00:10
- Location: The Independent Republic of Massachusetts
Re: How do you make Porteus so small?
Thanks.
Doing some size comparisons...
/usr/lib is not much larger on Debian than on Porteus Xfce 1.2 RC2. And /usr/bin is a little smaller on Debian.
/usr/share/locale is huge on Debian. On Porteus it's empty. Which makes sense, though complete removal of localization is IMO a heavy price.
/usr/share/doc is also fairly large on Debian, and mostly empty on Porteus.
It looks to me like running localepurge on Debian should solve most of the space wastage problem... Though again, I'm not entirely sure if it's worth it. I assume glibc locales can be reinstalled later?
Doing some size comparisons...
/usr/lib is not much larger on Debian than on Porteus Xfce 1.2 RC2. And /usr/bin is a little smaller on Debian.
/usr/share/locale is huge on Debian. On Porteus it's empty. Which makes sense, though complete removal of localization is IMO a heavy price.
/usr/share/doc is also fairly large on Debian, and mostly empty on Porteus.
It looks to me like running localepurge on Debian should solve most of the space wastage problem... Though again, I'm not entirely sure if it's worth it. I assume glibc locales can be reinstalled later?
- Ahau
- King of Docs
- Posts: 1331
- Joined: 28 Dec 2010, 15:18
- Distribution: LXDE & Xfce 32/64-bit
- Location: USA
Re: How do you make Porteus so small?
Yes, in an effort to minimize the size of the ISO, we strip the localization files, including glibc locales, and generate a "locales" module with those stripped files. If a user wants to have Porteus working in another language, they can use the Language-Selection-Tool, which allows the user to select their language, keyboard layout, fonts, etc,. and the tool downloads all the needed files and creates a custom module. Typically, we only have Language-Selection-Tool fully operational in a final release (not in development versions), but if you're running 1.2 RC2 you should still be able to run through the process and see how it operates.
What compression method are you using for your ISO?
What compression method are you using for your ISO?
Please take a look at our online documentation, here. Suggestions are welcome!
-
- White ninja
- Posts: 13
- Joined: 10 Jan 2012, 00:10
- Location: The Independent Republic of Massachusetts
Re: How do you make Porteus so small?
Compression method? Don't know. I would guess zlib or some such.
BTW, removing locales brought it down to 489 MB. Somewhat better but not ideal.
(Now let's see what clearing the doc files does...)
BTW, removing locales brought it down to 489 MB. Somewhat better but not ideal.
(Now let's see what clearing the doc files does...)
Re: How do you make Porteus so small?
lol, zlib is mostly used in PHP to compress data before to send them to browser and make "faster" the page loading and also increase the cpu load when you have a high traffic 

NjVFQzY2Rg==
-
- White ninja
- Posts: 13
- Joined: 10 Jan 2012, 00:10
- Location: The Independent Republic of Massachusetts
Re: How do you make Porteus so small?
Humm. Maybe gzip then?
Anyway stripping out the doc files brought it to 427 MB... And I really have to complement you guys on the job you did shrinking your distro. This is more difficult than I expected.
(If you're wondering what I'm up to, I'm trying to make something like Slax Popcorn Edition - only capable of a full hard drive install, with fully functional package management afterwards. I do suppose the Porteus/Slax module-based model may be better for usability in some ways, but in terms of security I would think it's better to have normal package management!)
Anyway stripping out the doc files brought it to 427 MB... And I really have to complement you guys on the job you did shrinking your distro. This is more difficult than I expected.

(If you're wondering what I'm up to, I'm trying to make something like Slax Popcorn Edition - only capable of a full hard drive install, with fully functional package management afterwards. I do suppose the Porteus/Slax module-based model may be better for usability in some ways, but in terms of security I would think it's better to have normal package management!)
- brokenman
- Site Admin
- Posts: 6104
- Joined: 27 Dec 2010, 03:50
- Distribution: Porteus v4 all desktops
- Location: Brazil
- Contact:
Re: How do you make Porteus so small?
Here is an example of some of the stuff that gets removed:
All the man pages left are gunzipped as lzma does a better job and all ELF symbols are reduced.
Then they are removed from the distro for inclusion in the language selection tool where users can download them as needed.
# move man files for export to LST
We leave all header files in place now, but used to remove them for inclusion in the 'development' module. These files (a.out and .h files) are really only needed for compiling stuff.
Finally all .a files are stripped and locales are exported for LST too.
I hope that helps a little in your mission.

Code: Select all
rm usr/man/man2/* 2>/dev/null
rm usr/man/man3/* 2>/dev/null
rm usr/man/man4/* 2>/dev/null
rm usr/man/man6/* 2>/dev/null
rm usr/man/man9/* 2>/dev/null
rm -r usr/share/gtk-2.0/demo
rm -r usr/share/gtk-doc
rm -rf /usr/doc/*
rm -rf /usr/share/doc/*
Code: Select all
find ./ -type d | xargs chmod -v 755 >/dev/null 2>&1
echo "Extracting All manpages..."
find . | grep .gz | xargs gunzip >/dev/null 2>&1
echo "stripping Unneeded Symbols..."
find . | xargs file | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded >/dev/null 2>&1
# move man files for export to LST
Code: Select all
[ -d $basef/$pver/export/man ] || mkdir -p $basef/$pver/export/man
for x in `ls usr/man | egrep -v "man|cat"`; do
cp -r usr/man/$x $basef/$pver/export/man
rm -rf usr/man/$x
done
Finally all .a files are stripped and locales are exported for LST too.
Code: Select all
echo "Stripping down ..."
find . -name "*.a" | xargs -n 1 strip -g
locales=$basef/$pver/export/
cp -r usr/share/locale $locales && rm -r usr/share/locale/*
Not sure what you mean by 'normal'. A good package manager should pull packages from a trusted source, resolve dependencies and make life easy for the user. That's what our package managers do ... our main manager does a little extra too.I would think it's better to have normal package management!

How do i become super user?
Wear your underpants on the outside and put on a cape.
Wear your underpants on the outside and put on a cape.
-
- White ninja
- Posts: 13
- Joined: 10 Jan 2012, 00:10
- Location: The Independent Republic of Massachusetts
Re: How do you make Porteus so small?
Thanks! 

But I would expect the combination of modules and (possibly overlapping) installable packages could potentially lead to all kinds of problems, no?brokenman wrote:Not sure what you mean by 'normal'. A good package manager should pull packages from a trusted source, resolve dependencies and make life easy for the user. That's what our package managers do ... our main manager does a little extra too.
- brokenman
- Site Admin
- Posts: 6104
- Joined: 27 Dec 2010, 03:50
- Distribution: Porteus v4 all desktops
- Location: Brazil
- Contact:
Re: How do you make Porteus so small?
If you are using the official porteus repo this won't be a problem as modules are kept up to date to prevent library clashes. If you are pulling from slackware/salix/debian/fido and mixing old and new then only the god of linux knows what could happen. So far i have not seen or had reported any problems in this area ... but for what it's worth your theory is correct.
How do i become super user?
Wear your underpants on the outside and put on a cape.
Wear your underpants on the outside and put on a cape.
- Ahau
- King of Docs
- Posts: 1331
- Joined: 28 Dec 2010, 15:18
- Distribution: LXDE & Xfce 32/64-bit
- Location: USA
Re: How do you make Porteus so small?
As a follow up on squashfs+xz compression, I'm messing with a module now (002-xorg.xzm for arm), which is 240MB when extracted, and 60MB when compressed...
Please take a look at our online documentation, here. Suggestions are welcome!