Porteus-5.0-rc1-Openbox with ZST compression

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
raja
Shogun
Shogun
Posts: 434
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and Porteus-Artix-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#1 by raja » 23 Jun 2019, 09:22

My first impressions to jss efforts.


Zst compression and deployment cuts down boot time by 3 seconds.

DMESG: kernel finishes in 6 seconds.

Code: Select all

[    0.000000] microcode: microcode updated early to revision 0xb4, date = 2019-04-01
[    0.000000] Linux version 5.1.13-porteus (root@porteus.example.net) (gcc version 8.3.0 (GCC)) #1 SMP PREEMPT Sat Jun 22 15:03:45 Local time zone must be set--
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-zst ipv6.disable=1 nomagic norootcopy

[    5.731644] USB Video Class driver (1.1.1)
[    5.777892] input: SYNA7DB5:00 06CB:7DB7 Touchpad as /devices/pci0000:00/0000:00:15.0/i2c_designware.0/i2c-0/i2c-SYNA7DB5:00/0018:06CB:7DB7.0001/input/input18
[    5.778414] hid-multitouch 0018:06CB:7DB7.0001: input,hidraw0: I2C HID v1.00 Mouse [SYNA7DB5:00 06CB:7DB7] on i2c-SYNA7DB5:00
[    5.850015] ath10k_pci 0000:02:00.0: board_file api 1 bmi_id N/A crc32 544289f7
[    5.925286] ath10k_pci 0000:02:00.0: unsupported HTC service id: 1536
[    5.943488] ath10k_pci 0000:02:00.0: htt-ver 3.56 wmi-op 4 htt-op 3 cal otp max-sta 32 raw 0 hwcrypto 1
[    6.004899] ath: EEPROM regdomain: 0x69
[    6.004902] ath: EEPROM indicates we should expect a direct regpair map
[    6.004906] ath: Country alpha2 being used: 00
[    6.004908] ath: Regpair used: 0x69
XORG INFO:

Code: Select all

[     7.723] 
X.Org X Server 1.20.5
X Protocol Version 11, Revision 0
[     7.724] Build Operating System: Slackware 15.0 Slackware Linux Project
[     7.724] Current Operating System: Linux porteus.example.net 5.1.13-porteus #1 SMP PREEMPT Sat Jun 22 15:03:45 Local time zone must be set-- x86_64
[     7.724] Kernel command line: quiet BOOT_IMAGE=/boot/vmlinuz-zst ipv6.disable=1 nomagic norootcopy

[     8.323] (**) Option "xkb_layout" "us"

Applications open much faster (instantaneous) and clicking your way through menu and reactions are a braze. This particular aspect will exhilarate many users.

Definitely, this experiment needs to be taken up seriously, by altering/adding scripts, whereever necessary.

Adding palemoon and usm-data modules.

Code: Select all

mksquashfs palemoon-28.5.2-x86_64-1 palemoon-28.5.2-x86_64-1.zstdm -b 1048576 -comp zstd -Xcompression-level 22
case1:

palemoon size-xzm : 41.0 MiB (43,036,672 bytes)
palemoon size- zst : 43.4 MiB (45,547,520 bytes)

case2:

usm-data size-xzm : 3.9 MiB (4,104,192 bytes)
usm-data size-zst : 3.8 MiB (4,034,560 bytes)

case2 zst file size is lower.

Happy with first impressions. Thanks jss. I will use this package at least once a day.

*jss, I did not check your dir2zstdm script. Did you use standard 15 compression level or changed to 22. Last time when I compressed a 300 MB file the increase was nominal with 22 level..
Linux Kernel-4.4.272 -32 bit; Linux Kernel-5.4.185 - 64 bit

jssouza
Legendary
Legendary
Posts: 1165
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm

Porteus-5.0-rc1-Openbox with ZST compression

Post#2 by jssouza » 24 Jun 2019, 04:58

raja wrote:
23 Jun 2019, 09:22
Did you use standard 15 compression level or changed to 22.
I used compression level of 22. I also kept the block size at 256K which is same as that of xz compression in dir2xzm. Dont know if a higher block size changes anything.

You can experiment with them by changing the -b 256K -Xcompression-level 22 options to mksquashfs in the /opt/porteus-scripts/dir2zstdm script in 001-core.

raja
Shogun
Shogun
Posts: 434
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and Porteus-Artix-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#3 by raja » 25 Jun 2019, 09:07

base modules size b=256k : 335.1MB

base modules size b=1M : 321.7

xorg size was 3 MB higher and it caused a delay of 1 sec. I will go with your module.

Porteus site proudly claims 15 seconds boot time and stability as its unique talking point.

A 100% reduction in boot time and fast deployment of applications with the new compression standard, makes it very special.

Progress means change. Change means something new to percieve, see,and feel. Otherwise v 3,4,5 or 6 for any application is old wine in new bottle, for an user., though significent efforts go into, from developers side.

zactivate,zdeactivate,xzm2zstdm, extract,build for zstd and listed in menu , would make this , a complete package.

I am thrilled and will use this, since I have built all essential modules, normally used by me.
Linux Kernel-4.4.272 -32 bit; Linux Kernel-5.4.185 - 64 bit

User avatar
Blaze
DEV Team
DEV Team
Posts: 3869
Joined: 28 Dec 2010, 11:31
Distribution: ⟰ Porteus current ☯ all DEs ☯
Location: ☭ Russian Federation, Lipetsk region, Dankov
Contact:

Porteus-5.0-rc1-Openbox with ZST compression

Post#4 by Blaze » 30 Jun 2019, 06:22

Anybody hear about LZ4 decompression?
https://www.phoronix.com/scan.php?page= ... Decompress
Linux 6.6.11-porteus #1 SMP PREEMPT_DYNAMIC Sun Jan 14 12:07:37 MSK 2024 x86_64 Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz GenuineIntel GNU/Linux
MS-7A12 » [AMD/ATI] Navi 23 [Radeon RX 6600] [1002:73ff] (rev c7) » Vengeance LPX 16GB DDR4 K2 3200MHz C16

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 3924
Joined: 20 Mar 2012, 03:42
Distribution: v5.0-64bit
Location: australia
Contact:

Porteus-5.0-rc1-Openbox with ZST compression

Post#5 by ncmprhnsbl » 30 Jun 2019, 08:16

Blaze wrote:
30 Jun 2019, 06:22
Anybody hear about LZ4 decompression?
interesting... https://github.com/lz4/lz4
LZ4 default ratio=2.101 compression=780 MB/s decompression=4970 MB/s
Zstandard(zstd) ratio=2.883 compression=515 MB/s decompression=1380 MB/s
appears squashfs-tools can be built with lz4 support.. https://github.com/plougher/squashfs-to ... er/INSTALL
raja wrote:
25 Jun 2019, 09:07
zactivate,zdeactivate,xzm2zstdm, extract,build for zstd and listed in menu , would make this , a complete package.
see here: Porteus with zstd compression (Post by ncmprhnsbl #72567) this should supply most of what you're looking for..
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

raja
Shogun
Shogun
Posts: 434
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and Porteus-Artix-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#6 by raja » 30 Jun 2019, 08:36

ncmprhnsbl ... Thanks for your ZTREE. Very much appreciate your quick response.
Linux Kernel-4.4.272 -32 bit; Linux Kernel-5.4.185 - 64 bit

User avatar
Blaze
DEV Team
DEV Team
Posts: 3869
Joined: 28 Dec 2010, 11:31
Distribution: ⟰ Porteus current ☯ all DEs ☯
Location: ☭ Russian Federation, Lipetsk region, Dankov
Contact:

Porteus-5.0-rc1-Openbox with ZST compression

Post#7 by Blaze » 30 Jun 2019, 09:20

Ok, waiting upcoming Linux 5.3 kernel with LZ4 in-place decompression and seems it will be the next longterm maintenance kernel.
Linux 6.6.11-porteus #1 SMP PREEMPT_DYNAMIC Sun Jan 14 12:07:37 MSK 2024 x86_64 Intel(R) Xeon(R) CPU E3-1270 v6 @ 3.80GHz GenuineIntel GNU/Linux
MS-7A12 » [AMD/ATI] Navi 23 [Radeon RX 6600] [1002:73ff] (rev c7) » Vengeance LPX 16GB DDR4 K2 3200MHz C16

User avatar
babam
Warlord
Warlord
Posts: 526
Joined: 16 Nov 2016, 10:30
Distribution: Porteus 5.0rc3 Xfce K6.1.1
Location: Rainy city

Porteus-5.0-rc1-Openbox with ZST compression

Post#8 by babam » 30 Jun 2019, 11:33

ncmprhnsbl wrote:
30 Jun 2019, 08:16
LZ4 default ratio=2.101 compression=780 MB/s decompression=4970 MB/s
Zstandard(zstd) ratio=2.883 compression=515 MB/s decompression=1380 MB/s
appears squashfs-tools can be built with lz4 support.. https://github.com/plougher/squashfs-to ... er/INSTALL
But the size of the .xzm modules ( LZ4 compression ) will be very large.
Sorry, my English is bad.

raja
Shogun
Shogun
Posts: 434
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and Porteus-Artix-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#9 by raja » 30 Jun 2019, 20:15

LZ4 support is there from Kernel-3.xxx and in squashfs tools.. I have seen a Debian variety asking you to choose compression, xz,LZ4 etc during remastering years ago. There is some confusion in that reporting.

Facebook reference is to ZSTD only. Compression ratio:2.884
Github reference to LZ4 lists, LZ4 HC -9 (v1.9.0) 2.721 with fairly good compression ratio may be new.
Linux Kernel-4.4.272 -32 bit; Linux Kernel-5.4.185 - 64 bit

User avatar
maia
Samurai
Samurai
Posts: 116
Joined: 08 Aug 2019, 16:56
Distribution: Porteus-5 64b Openbox
Location: Brasil

Porteus-5.0-rc1-Openbox with ZST compression

Post#10 by maia » 03 Nov 2019, 17:40

I tested here on a I5-3320M 8GB SSD ext2.
Full boot in 7 seconds, programs starting very fast. :)
Sorry my english is really, g##gle-tr4nsl4t0r

michalpelszyk
White ninja
White ninja
Posts: 14
Joined: 16 Apr 2020, 17:42
Distribution: Porteus OpenBox

Porteus-5.0-rc1-Openbox with ZST compression

Post#11 by michalpelszyk » 22 Apr 2020, 17:48

I will be a huge fan of zstd. I am playing around with older laptops (early Core2Duo era) and switching to zstd makes a measurable difference of about 25% reduction in boot time (from ~20 to ~15 between boot menu selection and desktop). Installed on the standard 80GB internal HDD.

I have installed this release on 2 computers already and it flies! :Yahoo!:

Even with a bunch of modules its under 30s.

Thank you all for the scripts above, they showed me how to create a mass converter. Including just in case anybody else finds it handy:

Code: Select all

#!/bin/bash
tempdirname="/tmp/zzmntmp"
mkdir $tempdirname
for f in *xzm
do
  mount -t squashfs -o ro,realtime $f $tempdirname
  mksquashfs $tempdirname $f.zstd  -b 1M -comp zstd -Xcompression-level 22
  umount $tempdirname
done
rmdir $tempdirname

Post Reply