Page 72 of 135
Porteus Kernel Builder
Posted: 06 Jan 2021, 10:08
by Blaze
babam, about F2FS
F2FS is a great file system. This is evidenced at least by the fact that, only after being born, it is already able to successfully compete with ext4, which has been licked for years and hundreds of people. However, do not expect miracles from her and run to format your device. Yes, many people claim a noticeable increase in performance even without tests, but again, most likely, these effects are caused by the same lazy write mechanism, and in fact there is no significant performance gain.
config
https://disk.admlr.lipetsk.ru/s/pwigBNKAatyqtzk
Porteus Kernel Builder
Posted: 06 Jan 2021, 10:21
by babam
Blaze, I like the compression feature.
Porteus Kernel Builder
Posted: 06 Jan 2021, 10:43
by maia
roadie wrote: ↑06 Jan 2021, 07:01
I just compiled a 5.10.4 for aufs using the patch from the site and am using it now. A bonus is that modules can be activated in the live system.
Work for me, thks roadie
Porteus Kernel Builder
Posted: 06 Jan 2021, 12:39
by babam
roadie wrote: ↑06 Jan 2021, 07:01
I just compiled a 5.10.4 for aufs using the patch from the site and am using it now. A bonus is that modules can be activated in the live system.
maia wrote: ↑06 Jan 2021, 10:43
Work for me, thks roadie
What are the advantages, is it faster?
Porteus Kernel Builder
Posted: 06 Jan 2021, 14:35
by maia
More pratical to apply, only a file with all the patches
Porteus Kernel Builder
Posted: 06 Jan 2021, 17:19
by roadie
babam wrote: ↑06 Jan 2021, 12:39
roadie wrote: ↑06 Jan 2021, 07:01
I just compiled a 5.10.4 for aufs using the patch from the site and am using it now. A bonus is that modules can be activated in the live system.
maia wrote: ↑06 Jan 2021, 10:43
Work for me, thks roadie
What are the advantages, is it faster?
As maia say's, it's easy and fast to apply, but I see it as another option for a patch when a catastrophe happens, and other patches aren't available.
Porteus Kernel Builder
Posted: 07 Jan 2021, 12:38
by Blaze
[Updated kernel Examples]
==== AUFS Kernel only for Porteus ====
Simple package (vmlinuz, 000-kernel.xzm, 06-crippled_sources-NNN-XXbit.xzm, 64bit.config)
[
5.10.5] <--
NEW : "All patches" patching was done.
64bit-ALL-kernel5.10.5.tar (~167 M)
ZSTD version
https://disk.admlr.lipetsk.ru/s/Bew33i4SaMejzeZ
5f0cf087d6f09052e3f69c1df29ffbe6 64bit-ALL-kernel5.10.5.tar
Note : Compiler that was used.
for kernel 5.10.5:
10.2.0-x86_64
Note. After compilation of kernel some my setting are gone.
Before:
Code: Select all
root@porteus:/tmp/linux-5.10.5# diff -Naur /home/guest/Загрузки/64bit.config /tmp/linux-5.10.5/.config
--- "/home/guest/\320\227\320\260\320\263\321\200\321\203\320\267\320\272\320\270/64bit.config" 2021-01-07 11:49:20.811384969 +0300
+++ /tmp/linux-5.10.5/.config 2021-01-07 12:54:04.238684024 +0300
@@ -1,6 +1,6 @@
#
# Automatically generated file; DO NOT EDIT.
-# Linux/x86 5.10.4 Kernel Configuration
+# Linux/x86 5.10.5 Kernel Configuration
#
CONFIG_CC_VERSION_TEXT="gcc (GCC) 10.2.0"
CONFIG_CC_IS_GCC=y
@@ -767,6 +767,10 @@
# end of GCOV-based kernel profiling
CONFIG_HAVE_GCC_PLUGINS=y
+CONFIG_GCC_PLUGINS=y
+# CONFIG_GCC_PLUGIN_CYC_COMPLEXITY is not set
+# CONFIG_GCC_PLUGIN_LATENT_ENTROPY is not set
+# CONFIG_GCC_PLUGIN_RANDSTRUCT is not set
# end of General architecture-dependent options
CONFIG_RT_MUTEXES=y
@@ -779,7 +783,10 @@
CONFIG_ASM_MODVERSIONS=y
# CONFIG_MODULE_SRCVERSION_ALL is not set
# CONFIG_MODULE_SIG is not set
-# CONFIG_MODULE_COMPRESS is not set
+CONFIG_MODULE_COMPRESS=y
+# CONFIG_MODULE_COMPRESS_GZIP is not set
+# CONFIG_MODULE_COMPRESS_XZ is not set
+CONFIG_MODULE_COMPRESS_ZSTD=y
CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=y
CONFIG_UNUSED_SYMBOLS=y
CONFIG_MODULES_TREE_LOOKUP=y
@@ -7997,7 +8004,10 @@
CONFIG_F2FS_FS_SECURITY=y
# CONFIG_F2FS_CHECK_FS is not set
# CONFIG_F2FS_FAULT_INJECTION is not set
-# CONFIG_F2FS_FS_COMPRESSION is not set
+CONFIG_F2FS_FS_COMPRESSION=y
+# CONFIG_F2FS_FS_LZO is not set
+CONFIG_F2FS_FS_LZ4=y
+CONFIG_F2FS_FS_ZSTD=y
CONFIG_FS_DAX=y
CONFIG_FS_POSIX_ACL=y
CONFIG_EXPORTFS=y
@@ -8312,6 +8322,10 @@
# Memory initialization
#
CONFIG_INIT_STACK_NONE=y
+# CONFIG_GCC_PLUGIN_STRUCTLEAK_USER is not set
+# CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF is not set
+# CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL is not set
+# CONFIG_GCC_PLUGIN_STACKLEAK is not set
# CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
# CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
# end of Memory initialization
@@ -8355,7 +8369,7 @@
CONFIG_CRYPTO_NULL2=y
# CONFIG_CRYPTO_PCRYPT is not set
CONFIG_CRYPTO_CRYPTD=m
-CONFIG_CRYPTO_AUTHENC=m
+CONFIG_CRYPTO_AUTHENC=y
# CONFIG_CRYPTO_TEST is not set
CONFIG_CRYPTO_SIMD=m
CONFIG_CRYPTO_GLUE_HELPER_X86=m
@@ -8616,7 +8630,7 @@
CONFIG_ZLIB_DEFLATE=y
CONFIG_LZO_COMPRESS=y
CONFIG_LZO_DECOMPRESS=y
-CONFIG_LZ4_COMPRESS=m
+CONFIG_LZ4_COMPRESS=y
CONFIG_LZ4HC_COMPRESS=m
CONFIG_LZ4_DECOMPRESS=y
CONFIG_ZSTD_COMPRESS=y
After:
Code: Select all
# diff /tmp/64bit.config /mnt/sdc1/kernelx64/v5.10.5/64/64bit.config
3c3
< # Linux/x86 5.10.4 Kernel Configuration
---
> # Linux/x86 5.10.5 Kernel Configuration
782c782,784
< # CONFIG_MODULE_COMPRESS is not set
---
> CONFIG_MODULE_COMPRESS=y
> CONFIG_MODULE_COMPRESS_GZIP=y
> # CONFIG_MODULE_COMPRESS_XZ is not set
8000c8002,8005
< # CONFIG_F2FS_FS_COMPRESSION is not set
---
> CONFIG_F2FS_FS_COMPRESSION=y
> # CONFIG_F2FS_FS_LZO is not set
> CONFIG_F2FS_FS_LZ4=y
> CONFIG_F2FS_FS_ZSTD=y
8358c8363
< CONFIG_CRYPTO_AUTHENC=m
---
> CONFIG_CRYPTO_AUTHENC=y
8619c8624
< CONFIG_LZ4_COMPRESS=m
---
> CONFIG_LZ4_COMPRESS=y
Thanks to
neko for his support.
Porteus Kernel Builder
Posted: 07 Jan 2021, 13:21
by babam
Blaze, Thank you very much for all your efforts in compiling the Kernel.
Please tell me what are the benefits of CONFIG_MODULE_COMPRESS=y and CONFIG_MODULE_COMPRESS_ZSTD=y?
Porteus Kernel Builder
Posted: 07 Jan 2021, 15:21
by Blaze
babam wrote: ↑07 Jan 2021, 13:21
Please tell me what are the benefits of CONFIG_MODULE_COMPRESS=y and CONFIG_MODULE_COMPRESS_ZSTD=y?
Seems nothing because my options:
Code: Select all
+CONFIG_MODULE_COMPRESS=y
+# CONFIG_MODULE_COMPRESS_GZIP is not set
+# CONFIG_MODULE_COMPRESS_XZ is not set
+CONFIG_MODULE_COMPRESS_ZSTD=y
after compilation convert to:
Code: Select all
# cat /usr/src/linux/.config | grep 'ZSTD\|MODULE_COMPRESS'
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_KERNEL_ZSTD=y
CONFIG_RD_ZSTD=y
CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y
# CONFIG_MODULE_COMPRESS_XZ is not set
CONFIG_F2FS_FS_ZSTD=y
CONFIG_SQUASHFS_ZSTD=y
# CONFIG_PSTORE_ZSTD_COMPRESS is not set
CONFIG_CRYPTO_ZSTD=m
CONFIG_ZSTD_COMPRESS=y
CONFIG_ZSTD_DECOMPRESS=y
CONFIG_DECOMPRESS_ZSTD=y
and I think these options:
Code: Select all
CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y
are must to gone. What do you think about this?
BTW you can try F2FS. I will try it too on my test USB flash drive
Porteus with ZSTD kernel boots faster, but the size is huge.
In your opinion, which is better size or boot up speed?
Porteus Kernel Builder
Posted: 07 Jan 2021, 16:17
by babam
Blaze wrote: ↑07 Jan 2021, 15:21
and I think these options:
Code: Select all
CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y
What's the benefits of that?
Blaze wrote: ↑07 Jan 2021, 15:21
BTW you can try F2FS. I will try it too on my test USB flash drive
Don't forget to provide the 'extra_ attr,compression' options at filesystem creation.
Code: Select all
# mkfs.f2fs -O extra_attr,compression /dev/sdXX
# mount -o compress_algorithm=zstd /dev/sdXX /mnt/sdXX
Blaze wrote: ↑07 Jan 2021, 15:21
Porteus with ZSTD kernel boots faster, but the size is huge.
In your opinion, which is better size or boot up speed?
Speed, i love speed.
Porteus Kernel Builder
Posted: 07 Jan 2021, 17:24
by Blaze
babam wrote: ↑07 Jan 2021, 16:17
What's the benefits of that?
No benefit, that's what I'm talking about above.
Grandfather GZIP is activated instead of CONFIG_MODULE_COMPRESS_ZSTD=y
Отправлено спустя 6 минут 40 секунд:
babam wrote: ↑07 Jan 2021, 16:17
Don't forget to provide the 'extra_ attr,compression' options at filesystem creation.
Info
https://wiki.archlinux.org/index.php/F2FS#Compression
not bad if this F2FS mount option add in fstab by default automatically and Gparted able to format with extra_attr,compression options
Porteus Kernel Builder
Posted: 07 Jan 2021, 18:10
by roadie
Blaze,
I'm not sure why your config isn't keeping the changes, I just built a kernel with xz compressed modules with no problems. It boots fine, but there's really very little difference in boot times between the gzip and xz compressed modules. I'll compile a kernel with uncompressed modules and check those times.
What patch are you using for zstd module compression in the kernel?
Code: Select all
stickbox:[guest]:~$ cd /tmp
stickbox:[guest]:/tmp$ ls
000-kernel-5.10.4-aufs-xz.xzm mc-root mloop palemoon_guest xdg-runtime-guest
stickbox:[guest]:/tmp$ lsxzm 000-kernel-5.10.4-aufs-xz.xzm
/lib/modules
/lib/modules/5.10.4-porteus
/lib/modules/5.10.4-porteus/build
/lib/modules/5.10.4-porteus/kernel
/lib/modules/5.10.4-porteus/kernel/arch
/lib/modules/5.10.4-porteus/kernel/arch/x86
/lib/modules/5.10.4-porteus/kernel/arch/x86/crypto
/lib/modules/5.10.4-porteus/kernel/arch/x86/crypto/aesni-intel.ko.xz
/lib/modules/5.10.4-porteus/kernel/arch/x86/crypto/crc32-pclmul.ko.xz
As for zstd, a patched kernel builds and boots, but the modules are not decompressed because
kmod needs to be patched for zstd. When this is done, the kernels can be loaded, but they don't get decompressed at boot because a
depmod-a command needs to be run first.....I don't know why this is.
I haven't found anywhere that I can run the depmod command in early boot, and the kernel modules aren't found until that's done. It may be a missing setting in the config that's causing this. I would think that if gzipped and xz compressed modules are found right away, zstd compressed should be as well, but that's not the case.
Porteus Kernel Builder
Posted: 07 Jan 2021, 19:39
by roadie
I just tried a kernel with uncompressed modules, and I really don't see much difference in boot time between uncompressed and xz or gz compressed.....maybe 1 second at best. I don't think it's worth the extra work to make zstd work with compressed modules, at least not for me.......but I probably will.....it's a new shiny thing and I want it.
Porteus Kernel Builder
Posted: 07 Jan 2021, 19:52
by Blaze
roadie wrote: ↑07 Jan 2021, 18:10
What patch are you using for zstd module compression in the kernel?
Damn, it's my fault. Don't laugh at me, but I didn't used the zstd module compression patches
Seems I must to use
0009-add-zstd-compressed-modules.patch
Code: Select all
diff --git a/Makefile b/Makefile
index f900c23b8291..387e83c4373e 100644
--- a/Makefile
+++ b/Makefile
@@ -971,8 +971,8 @@ endif # INSTALL_MOD_STRIP
export mod_strip_cmd
# CONFIG_MODULE_COMPRESS, if defined, will cause module to be compressed
-# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP
-# or CONFIG_MODULE_COMPRESS_XZ.
+# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP,
+# CONFIG_MODULE_COMPRESS_XZ, or CONFIG_MODULE_COMPRESS_ZSTD.
mod_compress_cmd = true
ifdef CONFIG_MODULE_COMPRESS
@@ -982,6 +982,9 @@ ifdef CONFIG_MODULE_COMPRESS
ifdef CONFIG_MODULE_COMPRESS_XZ
mod_compress_cmd = xz -f
endif # CONFIG_MODULE_COMPRESS_XZ
+ ifdef CONFIG_MODULE_COMPRESS_ZSTD
+ mod_compress_cmd = zstd -T0 -20 --ultra --rm -f
+ endif
endif # CONFIG_MODULE_COMPRESS
export mod_compress_cmd
diff --git a/init/Kconfig b/init/Kconfig
index a34064a031a5..345f068d0f62 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2102,8 +2102,8 @@ config MODULE_COMPRESS
bool "Compress modules on installation"
help
- Compresses kernel modules when 'make modules_install' is run; gzip or
- xz depending on "Compression algorithm" below.
+ Compresses kernel modules when 'make modules_install' is run; gzip,
+ xz, or zstd depending on "Compression algorithm" below.
module-init-tools MAY support gzip, and kmod MAY support gzip and xz.
@@ -2125,7 +2125,7 @@ choice
This determines which sort of compression will be used during
'make modules_install'.
- GZIP (default) and XZ are supported.
+ GZIP (default), XZ, and ZSTD are supported.
config MODULE_COMPRESS_GZIP
bool "GZIP"
@@ -2133,6 +2133,9 @@ config MODULE_COMPRESS_GZIP
config MODULE_COMPRESS_XZ
bool "XZ"
+config MODULE_COMPRESS_ZSTD
+ bool "ZSTD"
+
endchoice
config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS
roadie wrote: ↑07 Jan 2021, 18:10
but the modules are not decompressed because kmod needs to be patched for zstd. When this is done, the kernels can be loaded, but they don't get decompressed at boot because a depmod-a command needs to be run first.....I don't know why this is.
Code: Select all
module-init-tools MAY support gzip, and kmod MAY support gzip and xz.
in the 0009-add-zstd-compressed-modules.patch
Porteus Kernel Builder
Posted: 07 Jan 2021, 21:20
by roadie
Blaze,
I saw the warnings about kmod in the patch, but had to try anyway.....it's right, standard kmod doesn't work for zstd. I'm wondering if the 0009-add-zstd-compressed-modules.patch is all that's needed, maybe other patches are needed. The patched kmod does work to load the modules, but for some reason it isn't doing it automatically at boot.
I'm compiling another kernel right now with additional patches applied, I'll soon see if it makes a difference.