Slax-7 -> Porteus-2.0 backports

[fanthom]

after short discussion with other devs we have accepted few things from rather long list of changes made by Tomas M in his slax-7 development.

here they are:

1) squashfs blocksize for modules:
brokenman wants to stick with 256K for 32bits.
i'll do some testing with 128K and may go for it for all modules in 64bits as i prefer responsiveness/small memory footprint over the size (Ahau may recompress them back when not happy for XFCE edition), or maybe just for kde4, or drop it completely.
status: in testing for 64bits.

2) initramfs instead of initrd
status: acepted
todo: implemented already

3) Windows boot installer changes
status: partially accepted
todo:
a) VBS script for bypassing UAC (assigned: ?)
b) VBS script for discovering windows - dropped
c) installation to parition instead of MBR (assigned: myself)

4) Recompressing omni.ja zip archive for firefox
status: accepted
todo: implemented already

this list is not closed so we may add few things if community finds it useful.
please post your propositions for things which should be backported from Slax to Porteus with good argumentation as we have our contrary arguments prepared already.

Thanks.

[francois]

This is a very good idea to keep close to slax. After all, linux is linux and compatibility has a better taste.

Would it be useful to port porteus to android phones?
http://forum.porteus.org/viewtopic.php? ... 244#p11244

[brokenman]

brokenman had a 'momentary lapse of reason' and prefers responsiveness/small memory footprint over the size (128K)

[francois]

I am trying to follow you. Are'nt you the guy who told me sometime in the past that someday you were to build a mamoth size porteus?

[brokenman]

I don't recall making the comment, however this would be very simple to do. We just need to vote on the applications that should be included, and include them into a full size ISO. Anybody could do this.

[Ahau]

I think both myself and brokenman's comments to "stick with 256K block size" were in reference to a choice between 1MB and 256K, and I think we neglected to address the 128K proposal. I'm on board for 128K as well -- size matters, but performance trumps size (unless we're growing the ISO by 100MB to eek out 1MB of RAM and 1 second of boot time). It would be interesting to compare speed/RAM usage across all of the block sizes against module size. I feel a benchmarking script and a graph to show diminishing returns coming on...

[francois]

@fanthom:

Concerning trying to convince the father of slax to include and extensive array of driver, graphic and wifi. How much space is used in porteus? Citation from fanthom:
http://www.tomas-m.com/blog/19355-Slax- ... te-11.html
...
in Porteus is under 4MB and covers 95% of wifi chipsets...

In addition, how much space is required for graphic drivers in porteus?

[wread]

1) squashfs blocksize for modules:
brokenman wants to stick with 256K for 32bits.
i'll do some testing with 128K and may go for it for all modules in 64bits as i prefer responsiveness/small memory footprint over the size (Ahau may recompress them back when not happy for XFCE edition), or maybe just for kde4, or drop it completely.
status: in testing for 64bits.

With the actual block size squashing takes long with single-core old machines; with multi-core processors it is much faster. Unsquashing seems to be more important, then. I guess 128k gives a compacter module, but takes longer to unsquash, or am I wrong? If we go for 128k blocks, does it mean we will have to redo all modules? I hope not! At any rate all the versions should go for the same blocksize: either all bulls or all cows!

Ramfs:
What is the main difference between initrd (initialize ram disk) and initramfs (initialize ram file system); it is obvious they do the same, but please tell us why it is better. Thanks, Tom.

Regards!

[Hamza]

@wread,

If we switch from 256k to 128k, this should be faster to unsquash normally.

[wread]

@Hamza
Dont be so sure; you will have to unsquash twice so many 128k-blocks as 256k-blocks, and the iteration swallows an important amount of time (that's the trick by the "boosts" modules!). Let's wait for results of fanthom tests....

Regards

[fanthom]

Concerning trying to convince the father of slax to include and extensive array of driver

i have actually given up. it's hard to convince Tomas to anything... time will tell who was right in this case

in Porteus is under 4MB and covers 95% of wifi chipsets...

just checked and current 'extra-firmware' package for porteus-2.0 is 3.8MB
will also include boradcom-sta by default so we should have the same cover as in porteus-1.2 for wifi.

how much space is required for graphic drivers in porteus?

i think you are referring to a firmware required by radeon driver (other drivers have fw compiled in). it's 650KB uncompressed and always was included in Porteus. this is not going to change...

I guess 128k gives a compacter module

higher block size means smaller module but uses more memory to mount xzm and more CPU power to read from it.

here is how i understand the "block size" issue:
let's say you have a module containing small files only: 000-kernel.xzm is a perfect example cause most kernel drivers are about 100KB in size.
let's say udev is going to load 15 drivers from 000-kernel.xzm during boot time.
if you compress a module with 1MB block size and these drivers will be spread over whole xzm (worst scenario) then udev will have to read 15MB of data just to unpack 1.5MB of drivers. that's a huge waste of I/O thus CPU power (and causing slow boot).

if you compress a module with 128K block size (or even better 64K) then probably only about 2MB of data will have to be uncompressed to get an access to these 15 drivers.

my assumptions seems to be correct, let me qute Phillip Lougher (a squashfs creator):

Code: Select all

The smaller the buffer cache I/O block the less wasted I/O is performed.
For instance a Squashfs block overlapping 60 bytes into the last buffer
cache block @ 1K I/O size results in less wasted I/O than @ 4K I/O size. 

and

Code: Select all

smaller buffer cache block size results in less I/O performed by Squashfs

reference:
link

i have done my own tests:
a) i have reduced block size in all modules from porteus-2.0 alpha from 256KB (default) to 128KB
b) attached one 3.0Ghz core to vbox session and reduced it's execution cap to 33%. this way i got env similar to pentium II 1000 Mhz.

result:
10% faster boot when using 128KB block size.

the drawback of this test was that it measured only CPU time needed for getting to the desktop. in vbox case reading speed was unlimited which is not a "real life" scenario.

if you take for example i7 CPU and some slow usb stick then higher block size can result in faster booting as usb reading speed would be a bottleneck and not the CPU power.

on the other hand if Porteus would be installed on a SSD drive with weak CPU (netbook with Atom CPU) then lower block size should be a pure winner.

another factor is a power needed for certain operations (this affects battery life).
lower block size is a winner here as less I/O means less CPU power needed to uncompress required data.
lower block size == longer battery life

i hope all it sounds reasonable

it's really hard to make a decision regarding default bs for modules in porteus-2.0.
many factors to take into account.

i think we could stick to default 256KB as user can always recompress the modules for lower block sizes (or even gzip/lzo compression) when needed.

If we go for 128k blocks, does it mean we will have to redo all modules?

nope - block size and even compression algorithm can be different. porteus kernel will accept them all.

What is the main difference between initrd (initialize ram disk) and initramfs (initialize ram file system); it is obvious they do the same, but please tell us why it is better

here they are:
a) initramfs can have any size (1MB, 10MB) and does not require providing a cheatcode to the kernel to tell about its size (initrd required 'ramdisk_size=some_value')
b) in porteus-2,0 case initramfs will be unpacked on tmpfs (RAM filesystem) so at the end on booting process unused data (binaries/libraries) can be deleted from it to free up the memory. initrd was taking full 2.6 MB of ram (6.6MB in slax-6.1.2 case), initramfs in porteus-2.0 should'n take more than 300KB as we need only busybox for shutdown, the rest can be deleted.
c) tmpfs mounted in /mnt/live folder can be very handy as we can use it from porteus directly. for example i have linked /run folder to /mnt/live/run (so i dont have to mount tmpfs twice and also blkid cache can be shared) and exported KDEVARTMP to /mnt/live/var/tmp so big kde4 cache files are always placed in RAM even when booting with 'changes=' cheatcode. this trick should benefit kde4 users (who are saving changes) in faster boot/overall desktop performance.
d) you can use multiple initramfs as they work in a layered structure (same as aufs+modules). this way we are going to have 2 initramfs in porteus-2.0: main one holding all utils/libs/linuxrc script and second one holding all drivers needed for pxe-boot (porteus-2.0 is going to have full support for all ethernet devices). second initrd will be loaded only by pxe clients so booting to normal session wont be slowed down even by a millisecond (btw - booting and shutdown speed of porteus-2.0 is much improved again, more to come in official porteus-2.0 changelog).

as far as i remember these are main differences in initramfs vs initrd battle.

Cheers

[francois]

Thanks a lot for these illuminating precisions.

[stalane]

Great work. I am on the fence vis-vis block size because its a trade off. But initramfs I agree is the way to go.

[Quax]

Hi Fanthom,

...it's hard to convince Tomas to anything...

I've made the experience, that he has changed his mind in some areas

It was fairly easy to convince him to (improve and) use zram.
Same was with my detection scripts for Poulsbo chipsets.
Even while he invented dynfilefs, at the end he integrated ALL my requests for extensions.

Last but not least: The collaboration while adapting/improving his windows detection scripts was a real pleasure

Quax

[fanthom]

@Quax

you are right.
i'll rephrase my statement:

..it's hard to convince Tomas to anything what brings additional size to the slax ISO