Porteus with zstd compression

[jssouza]

Here's Porteus v5.0rc1 openbox remastered with zstd compression.
http://www.mediafire.com/file/zwndkxavp ... d.iso/file
md5sum: 54c88d7d8594872471e10cea95f3629d

- kernel built with zstd support for squashfs
- initrd loads only zstdm modules
- 000-kernel.zstdm, 001-core.zstdm, 002-xorg.zstdm and 003-openbox.zstdm modules squashed with zstd compression
- Some scripts added in 001-core: zactivate, zdeactivate, dir2zstdm (command line only) for zstd module creation, activation and deactivation (not fully tested)

I get a boot time of 5 seconds in virtualbox.

@raja please note, if you plan to remaster, you need the 000-kernel.zstdm, vmlinuz as well as initrd.xz.

[fulalas]

Pardon my ignorance, but what's the actual difference between this and the official release?

[ncmprhnsbl]

Pardon my ignorance, but what's the actual difference between this and the official release?

see discussion here: ZSTD compression instead of XZ for the modules
basicly, using zstd compression instead of xz ... advantage: faster decompression disadvantage: larger module size

[jssouza]

I also noticed that there is no support (yet) for compressing kernel and the initrd itself in zstd format. Not sure if this is not possible or might come later in the kernel. I remember xz compression of the kernel also came later, after squashfs xz support had arrived. In old computers, uncompressing the kernel itself takes a long time.
Of course boot loaders (grub, syslinux) would also need to support loading zstd compressed vmlinuz and initrd.

[babam]

I get a boot time of 5 seconds in virtualbox.

What if installed on NTFS, is it still fast.

[jssouza]

What if installed on NTFS, is it still fast.

I did not see any noticeable difference. Boot from ntfs also had same boot time and same response times.

[Kulle]

Hi,
is a change over from xz to zstd useful just because it is slightly faster to decompress?
I think, what has been tried and tested for a long time should be maintained.
And you can continue to use existing xzm modules.

[ncmprhnsbl]

zactivate needs an edit to aufs-insert to work:
@ line 18 (changing .xzm to .zstdm)

Code: Select all

if [[ $(echo "$BASE" | grep .zstdm) = "" ]]; then
    echo "$BASE: Module must end with .zstdm"
    exit 1
fi

of course, simply renaming the (zstd)module *.xzm would work too since unsquashfs detects the protocol, whatever it is..
probably pointing zactivate at an extra "zaufs-insert" script is better(and maybe zxactivate).. then not breaking activate..
what might be interesting, would be to expand the present scripts (and initrd) to handle multiple compression types..
but hey, they already do, provided the modules are named *.xzm (apart from dir2xzm)
perhaps all we need is dir2zstdm that outputs *-zstd.xzm ...
furthermore, modules can be any of: gzip lzma lzo xz(what we use) or zstd .. provided they are named *.xzm they can be activated without modifying porteus..

[jssouza]

zactivate needs an edit to aufs-insert to work:

Check again, zactivate calls aufs-zinsert

[ncmprhnsbl]

Check again, zactivate calls aufs-zinsert

sorry, missed that... but it failed with the "must be .xzm" message .. something else going on..
EDIT:
ok.. this line in xactivate:

Code: Select all

# Insert the module to live filesystem: (changed to)
/opt/porteus-scripts/xorg/aufs-zinsert "$1"

but as i say above none of this is really necessary, if the standard module extension(.xzm) is used..

[jssouza]

It's true, agree with you, but when one facilitates an iso as an experiment, I would think both xzm and zstdm scripts should be present for comparison dont you think?

That is why I compiled the kernel with both xzm as well as zstdm support, and, also instead of overwriting the existing scripts, I (tried) created new zstd variants of them, alongside the xzm scripts.

ok.. this line in xactivate:

Will try and fix this. Thanks ncmprhnsbl.

[jssouza]

Hi,
is a change over from xz to zstd useful just because it is slightly faster to decompress?
I think, what has been tried and tested for a long time should be maintained.
And you can continue to use existing xzm modules.

I almost agree with you Kulle.
But, here's the thing:
When lzma1 was the thing and lzma2 just came out, Tomas M was just preparing Slax 7, and I in my ignorant beginning just mentioned, why still lzma1 on the kernel? Why not xz on it too?
Tomas replied, Tomas acknowledged me, and changed the kernel to xz. Porteus changed kernel to xz soon. Post is available for ppl who search good enough.

Coming to the point: I made a difference then, both in slax and in porteus. Thought, why not let the community try it again?

[ottone]

I have to say that on my old Acer Laptop ( T7500) this iso is monsterfast . By the way , the kernel 5xx alone on the normal isos makes a great difference in speed

on my old lappy - surprising ....

[ncmprhnsbl]

It's true, agree with you, but when one facilitates an iso as an experiment, I would think both xzm and zstdm scripts should be present for comparison dont you think?

absolutely, if zstd is to be incorporated officially, i'd like to see it provided alongside xz, so users can choose for themselves according to their preference for speed or size..
here's what i've done:
swapped your zstd/xz enabled kernel into my standard install - but not your initrd - so renamed to 000-kernel-zstd.xzm
so: .xzms and -zstd.xzms will activate at boot, and a lot of the gui scripts will work oftb .ie activate/deactivate/extract
just had change dir2stdm to output *zstd.xzm, add /opt/porteus-scripts/context-menu/create-zmodule and a corresponding .desktop..

i'm curious about the kernel support for compressors, is this part of the AUFS patching or some other compile flag .. how (not)trivial is this?
i remember back when alphaos was still alive, simargl switched to gzip for a speed increase, i wonder if it might be interesting to play with too, tho size takes a hit..

[jssouza]

Great!

i'm curious about the kernel support for compressors, is this part of the AUFS patching or some other compile flag .. how (not)trivial is this?

Nope, no aufs patching is necessary. In fact, I think it is now already part of the kernel
@raja requested here Porteus Kernel Builder (Post by raja #72503) and @neko added support in the next post. I had added some extra zstd related configs to my kernel in the iso, but I dont think they are necessary. Just CONFIG_SQUASHFS_ZSTD=y should be enough.

i remember back when alphaos was still alive, simargl switched to gzip for a speed increase, i wonder if it might be interesting to play with too, tho size takes a hit..

I had done similar experiments by creating lzo modules in the raspberry pi variant, to increase speed. The modules did increase in size a lot, but speed increase was quite noticeable.
zlib, lz4, lzo squashfs compressors are already supported in our kernel.