Porteus-5.0-rc1-Openbox with ZST compression

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
Post Reply
raja
Shogun
Shogun
Posts: 213
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and v5.0-rc1-zstd-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#1 by raja » 23 Jun 2019, 09:22

My first impressions to jss efforts.


Zst compression and deployment cuts down boot time by 3 seconds.

DMESG: kernel finishes in 6 seconds.

Code: Select all

[    0.000000] microcode: microcode updated early to revision 0xb4, date = 2019-04-01
[    0.000000] Linux version 5.1.13-porteus (root@porteus.example.net) (gcc version 8.3.0 (GCC)) #1 SMP PREEMPT Sat Jun 22 15:03:45 Local time zone must be set--
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-zst ipv6.disable=1 nomagic norootcopy

[    5.731644] USB Video Class driver (1.1.1)
[    5.777892] input: SYNA7DB5:00 06CB:7DB7 Touchpad as /devices/pci0000:00/0000:00:15.0/i2c_designware.0/i2c-0/i2c-SYNA7DB5:00/0018:06CB:7DB7.0001/input/input18
[    5.778414] hid-multitouch 0018:06CB:7DB7.0001: input,hidraw0: I2C HID v1.00 Mouse [SYNA7DB5:00 06CB:7DB7] on i2c-SYNA7DB5:00
[    5.850015] ath10k_pci 0000:02:00.0: board_file api 1 bmi_id N/A crc32 544289f7
[    5.925286] ath10k_pci 0000:02:00.0: unsupported HTC service id: 1536
[    5.943488] ath10k_pci 0000:02:00.0: htt-ver 3.56 wmi-op 4 htt-op 3 cal otp max-sta 32 raw 0 hwcrypto 1
[    6.004899] ath: EEPROM regdomain: 0x69
[    6.004902] ath: EEPROM indicates we should expect a direct regpair map
[    6.004906] ath: Country alpha2 being used: 00
[    6.004908] ath: Regpair used: 0x69
XORG INFO:

Code: Select all

[     7.723] 
X.Org X Server 1.20.5
X Protocol Version 11, Revision 0
[     7.724] Build Operating System: Slackware 15.0 Slackware Linux Project
[     7.724] Current Operating System: Linux porteus.example.net 5.1.13-porteus #1 SMP PREEMPT Sat Jun 22 15:03:45 Local time zone must be set-- x86_64
[     7.724] Kernel command line: quiet BOOT_IMAGE=/boot/vmlinuz-zst ipv6.disable=1 nomagic norootcopy

[     8.323] (**) Option "xkb_layout" "us"

Applications open much faster (instantaneous) and clicking your way through menu and reactions are a braze. This particular aspect will exhilarate many users.

Definitely, this experiment needs to be taken up seriously, by altering/adding scripts, whereever necessary.

Adding palemoon and usm-data modules.

Code: Select all

mksquashfs palemoon-28.5.2-x86_64-1 palemoon-28.5.2-x86_64-1.zstdm -b 1048576 -comp zstd -Xcompression-level 22
case1:

palemoon size-xzm : 41.0 MiB (43,036,672 bytes)
palemoon size- zst : 43.4 MiB (45,547,520 bytes)

case2:

usm-data size-xzm : 3.9 MiB (4,104,192 bytes)
usm-data size-zst : 3.8 MiB (4,034,560 bytes)

case2 zst file size is lower.

Happy with first impressions. Thanks jss. I will use this package at least once a day.

*jss, I did not check your dir2zstdm script. Did you use standard 15 compression level or changed to 22. Last time when I compressed a 300 MB file the increase was nominal with 22 level..
Linux Kernel-4.4.120-32 bit; Linux kernel-5.2-64 bit.

jssouza
DEV Team
DEV Team
Posts: 1016
Joined: 09 Jul 2015, 14:17
Distribution: Porteus x86 arm
Location: Liechtenstein

Porteus-5.0-rc1-Openbox with ZST compression

Post#2 by jssouza » 24 Jun 2019, 04:58

raja wrote:
23 Jun 2019, 09:22
Did you use standard 15 compression level or changed to 22.
I used compression level of 22. I also kept the block size at 256K which is same as that of xz compression in dir2xzm. Dont know if a higher block size changes anything.

You can experiment with them by changing the -b 256K -Xcompression-level 22 options to mksquashfs in the /opt/porteus-scripts/dir2zstdm script in 001-core.

raja
Shogun
Shogun
Posts: 213
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and v5.0-rc1-zstd-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#3 by raja » 25 Jun 2019, 09:07

base modules size b=256k : 335.1MB

base modules size b=1M : 321.7

xorg size was 3 MB higher and it caused a delay of 1 sec. I will go with your module.

Porteus site proudly claims 15 seconds boot time and stability as its unique talking point.

A 100% reduction in boot time and fast deployment of applications with the new compression standard, makes it very special.

Progress means change. Change means something new to percieve, see,and feel. Otherwise v 3,4,5 or 6 for any application is old wine in new bottle, for an user., though significent efforts go into, from developers side.

zactivate,zdeactivate,xzm2zstdm, extract,build for zstd and listed in menu , would make this , a complete package.

I am thrilled and will use this, since I have built all essential modules, normally used by me.
Linux Kernel-4.4.120-32 bit; Linux kernel-5.2-64 bit.

User avatar
Blaze
DEV Team
DEV Team
Posts: 2624
Joined: 28 Dec 2010, 11:31
Distribution: ⟰ Porteus current ☯ all DEs ☯
Location: ☭ Russian Federation, Lipetsk region, Dankov
Contact:

Porteus-5.0-rc1-Openbox with ZST compression

Post#4 by Blaze » 30 Jun 2019, 06:22

Anybody hear about LZ4 decompression?
https://www.phoronix.com/scan.php?page= ... Decompress
Linux 5.3.0-rc3-porteus #1 SMP Mon Aug 5 11:28:55 UTC 2019 x86_64 Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz GenuineIntel GNU/Linux
MS-7A12 » [AMD/ATI] Tobago PRO [Radeon R7 360 / R9 360 OEM] (rev 81) » Vengeance LPX 16GB DDR4 K2 3200MHz C16

User avatar
ncmprhnsbl
DEV Team
DEV Team
Posts: 2178
Joined: 20 Mar 2012, 03:42
Distribution: 5.0rc1-64bit all-DE+more
Location: australia
Contact:

Porteus-5.0-rc1-Openbox with ZST compression

Post#5 by ncmprhnsbl » 30 Jun 2019, 08:16

Blaze wrote:
30 Jun 2019, 06:22
Anybody hear about LZ4 decompression?
interesting... https://github.com/lz4/lz4
LZ4 default ratio=2.101 compression=780 MB/s decompression=4970 MB/s
Zstandard(zstd) ratio=2.883 compression=515 MB/s decompression=1380 MB/s
appears squashfs-tools can be built with lz4 support.. https://github.com/plougher/squashfs-to ... er/INSTALL
raja wrote:
25 Jun 2019, 09:07
zactivate,zdeactivate,xzm2zstdm, extract,build for zstd and listed in menu , would make this , a complete package.
see here: Porteus with zstd compression (Post by ncmprhnsbl #72567) this should supply most of what you're looking for..
Forum Rules : https://forum.porteus.org/viewtopic.php?f=35&t=44

raja
Shogun
Shogun
Posts: 213
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and v5.0-rc1-zstd-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#6 by raja » 30 Jun 2019, 08:36

ncmprhnsbl ... Thanks for your ZTREE. Very much appreciate your quick response.
Linux Kernel-4.4.120-32 bit; Linux kernel-5.2-64 bit.

User avatar
Blaze
DEV Team
DEV Team
Posts: 2624
Joined: 28 Dec 2010, 11:31
Distribution: ⟰ Porteus current ☯ all DEs ☯
Location: ☭ Russian Federation, Lipetsk region, Dankov
Contact:

Porteus-5.0-rc1-Openbox with ZST compression

Post#7 by Blaze » 30 Jun 2019, 09:20

Ok, waiting upcoming Linux 5.3 kernel with LZ4 in-place decompression and seems it will be the next longterm maintenance kernel.
Linux 5.3.0-rc3-porteus #1 SMP Mon Aug 5 11:28:55 UTC 2019 x86_64 Intel(R) Core(TM) i5-6600K CPU @ 3.50GHz GenuineIntel GNU/Linux
MS-7A12 » [AMD/ATI] Tobago PRO [Radeon R7 360 / R9 360 OEM] (rev 81) » Vengeance LPX 16GB DDR4 K2 3200MHz C16

babam
Black ninja
Black ninja
Posts: 64
Joined: 16 Nov 2016, 10:30
Distribution: Porteus 3.2.2
Location: Rainy city

Porteus-5.0-rc1-Openbox with ZST compression

Post#8 by babam » 30 Jun 2019, 11:33

ncmprhnsbl wrote:
30 Jun 2019, 08:16
LZ4 default ratio=2.101 compression=780 MB/s decompression=4970 MB/s
Zstandard(zstd) ratio=2.883 compression=515 MB/s decompression=1380 MB/s
appears squashfs-tools can be built with lz4 support.. https://github.com/plougher/squashfs-to ... er/INSTALL
But the size of the .xzm modules ( LZ4 compression ) will be very large.
Sorry, my English is bad.

raja
Shogun
Shogun
Posts: 213
Joined: 02 May 2017, 09:51
Distribution: v3.2.2-32 and v5.0-rc1-zstd-64
Location: Chennai,India

Porteus-5.0-rc1-Openbox with ZST compression

Post#9 by raja » 30 Jun 2019, 20:15

LZ4 support is there from Kernel-3.xxx and in squashfs tools.. I have seen a Debian variety asking you to choose compression, xz,LZ4 etc during remastering years ago. There is some confusion in that reporting.

Facebook reference is to ZSTD only. Compression ratio:2.884
Github reference to LZ4 lists, LZ4 HC -9 (v1.9.0) 2.721 with fairly good compression ratio may be new.
Linux Kernel-4.4.120-32 bit; Linux kernel-5.2-64 bit.

Post Reply