Porteus Kernel Builder

[Blaze]

babam, about F2FS

F2FS is a great file system. This is evidenced at least by the fact that, only after being born, it is already able to successfully compete with ext4, which has been licked for years and hundreds of people. However, do not expect miracles from her and run to format your device. Yes, many people claim a noticeable increase in performance even without tests, but again, most likely, these effects are caused by the same lazy write mechanism, and in fact there is no significant performance gain.

config https://disk.admlr.lipetsk.ru/s/pwigBNKAatyqtzk

[babam]

Blaze, I like the compression feature.

[maia]

I just compiled a 5.10.4 for aufs using the patch from the site and am using it now. A bonus is that modules can be activated in the live system.

Work for me, thks roadie

[babam]

I just compiled a 5.10.4 for aufs using the patch from the site and am using it now. A bonus is that modules can be activated in the live system.

Work for me, thks roadie

What are the advantages, is it faster?

[maia]

More pratical to apply, only a file with all the patches

[roadie]

I just compiled a 5.10.4 for aufs using the patch from the site and am using it now. A bonus is that modules can be activated in the live system.

Work for me, thks roadie

What are the advantages, is it faster?

As maia say's, it's easy and fast to apply, but I see it as another option for a patch when a catastrophe happens, and other patches aren't available.

[Blaze]

[Updated kernel Examples]
==== AUFS Kernel only for Porteus ====
Simple package (vmlinuz, 000-kernel.xzm, 06-crippled_sources-NNN-XXbit.xzm, 64bit.config)

[5.10.5] <-- NEW : "All patches" patching was done.
64bit-ALL-kernel5.10.5.tar (~167 M) ZSTD version
https://disk.admlr.lipetsk.ru/s/Bew33i4SaMejzeZ
5f0cf087d6f09052e3f69c1df29ffbe6 64bit-ALL-kernel5.10.5.tar

Note : Compiler that was used.
for kernel 5.10.5: 10.2.0-x86_64

Note. After compilation of kernel some my setting are gone.

Before:

Code: Select all

root@porteus:/tmp/linux-5.10.5# diff -Naur /home/guest/Загрузки/64bit.config /tmp/linux-5.10.5/.config
--- "/home/guest/\320\227\320\260\320\263\321\200\321\203\320\267\320\272\320\270/64bit.config"	2021-01-07 11:49:20.811384969 +0300
+++ /tmp/linux-5.10.5/.config	2021-01-07 12:54:04.238684024 +0300
@@ -1,6 +1,6 @@
 #
 # Automatically generated file; DO NOT EDIT.
-# Linux/x86 5.10.4 Kernel Configuration
+# Linux/x86 5.10.5 Kernel Configuration
 #
 CONFIG_CC_VERSION_TEXT="gcc (GCC) 10.2.0"
 CONFIG_CC_IS_GCC=y
@@ -767,6 +767,10 @@
 # end of GCOV-based kernel profiling
 
 CONFIG_HAVE_GCC_PLUGINS=y
+CONFIG_GCC_PLUGINS=y
+# CONFIG_GCC_PLUGIN_CYC_COMPLEXITY is not set
+# CONFIG_GCC_PLUGIN_LATENT_ENTROPY is not set
+# CONFIG_GCC_PLUGIN_RANDSTRUCT is not set
 # end of General architecture-dependent options
 
 CONFIG_RT_MUTEXES=y
@@ -779,7 +783,10 @@
 CONFIG_ASM_MODVERSIONS=y
 # CONFIG_MODULE_SRCVERSION_ALL is not set
 # CONFIG_MODULE_SIG is not set
-# CONFIG_MODULE_COMPRESS is not set
+CONFIG_MODULE_COMPRESS=y
+# CONFIG_MODULE_COMPRESS_GZIP is not set
+# CONFIG_MODULE_COMPRESS_XZ is not set
+CONFIG_MODULE_COMPRESS_ZSTD=y
 CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS=y
 CONFIG_UNUSED_SYMBOLS=y
 CONFIG_MODULES_TREE_LOOKUP=y
@@ -7997,7 +8004,10 @@
 CONFIG_F2FS_FS_SECURITY=y
 # CONFIG_F2FS_CHECK_FS is not set
 # CONFIG_F2FS_FAULT_INJECTION is not set
-# CONFIG_F2FS_FS_COMPRESSION is not set
+CONFIG_F2FS_FS_COMPRESSION=y
+# CONFIG_F2FS_FS_LZO is not set
+CONFIG_F2FS_FS_LZ4=y
+CONFIG_F2FS_FS_ZSTD=y
 CONFIG_FS_DAX=y
 CONFIG_FS_POSIX_ACL=y
 CONFIG_EXPORTFS=y
@@ -8312,6 +8322,10 @@
 # Memory initialization
 #
 CONFIG_INIT_STACK_NONE=y
+# CONFIG_GCC_PLUGIN_STRUCTLEAK_USER is not set
+# CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF is not set
+# CONFIG_GCC_PLUGIN_STRUCTLEAK_BYREF_ALL is not set
+# CONFIG_GCC_PLUGIN_STACKLEAK is not set
 # CONFIG_INIT_ON_ALLOC_DEFAULT_ON is not set
 # CONFIG_INIT_ON_FREE_DEFAULT_ON is not set
 # end of Memory initialization
@@ -8355,7 +8369,7 @@
 CONFIG_CRYPTO_NULL2=y
 # CONFIG_CRYPTO_PCRYPT is not set
 CONFIG_CRYPTO_CRYPTD=m
-CONFIG_CRYPTO_AUTHENC=m
+CONFIG_CRYPTO_AUTHENC=y
 # CONFIG_CRYPTO_TEST is not set
 CONFIG_CRYPTO_SIMD=m
 CONFIG_CRYPTO_GLUE_HELPER_X86=m
@@ -8616,7 +8630,7 @@
 CONFIG_ZLIB_DEFLATE=y
 CONFIG_LZO_COMPRESS=y
 CONFIG_LZO_DECOMPRESS=y
-CONFIG_LZ4_COMPRESS=m
+CONFIG_LZ4_COMPRESS=y
 CONFIG_LZ4HC_COMPRESS=m
 CONFIG_LZ4_DECOMPRESS=y
 CONFIG_ZSTD_COMPRESS=y

After:

Code: Select all

# diff /tmp/64bit.config /mnt/sdc1/kernelx64/v5.10.5/64/64bit.config
3c3
< # Linux/x86 5.10.4 Kernel Configuration
---
> # Linux/x86 5.10.5 Kernel Configuration
782c782,784
< # CONFIG_MODULE_COMPRESS is not set
---
> CONFIG_MODULE_COMPRESS=y
> CONFIG_MODULE_COMPRESS_GZIP=y
> # CONFIG_MODULE_COMPRESS_XZ is not set
8000c8002,8005
< # CONFIG_F2FS_FS_COMPRESSION is not set
---
> CONFIG_F2FS_FS_COMPRESSION=y
> # CONFIG_F2FS_FS_LZO is not set
> CONFIG_F2FS_FS_LZ4=y
> CONFIG_F2FS_FS_ZSTD=y
8358c8363
< CONFIG_CRYPTO_AUTHENC=m
---
> CONFIG_CRYPTO_AUTHENC=y
8619c8624
< CONFIG_LZ4_COMPRESS=m
---
> CONFIG_LZ4_COMPRESS=y

Thanks to neko for his support.

[babam]

Blaze, Thank you very much for all your efforts in compiling the Kernel.

Please tell me what are the benefits of CONFIG_MODULE_COMPRESS=y and CONFIG_MODULE_COMPRESS_ZSTD=y?

[Blaze]

Please tell me what are the benefits of CONFIG_MODULE_COMPRESS=y and CONFIG_MODULE_COMPRESS_ZSTD=y?

Seems nothing because my options:

Code: Select all

+CONFIG_MODULE_COMPRESS=y
+# CONFIG_MODULE_COMPRESS_GZIP is not set
+# CONFIG_MODULE_COMPRESS_XZ is not set
+CONFIG_MODULE_COMPRESS_ZSTD=y

after compilation convert to:

Code: Select all

# cat /usr/src/linux/.config | grep 'ZSTD\|MODULE_COMPRESS'
CONFIG_HAVE_KERNEL_ZSTD=y
CONFIG_KERNEL_ZSTD=y
CONFIG_RD_ZSTD=y
CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y
# CONFIG_MODULE_COMPRESS_XZ is not set
CONFIG_F2FS_FS_ZSTD=y
CONFIG_SQUASHFS_ZSTD=y
# CONFIG_PSTORE_ZSTD_COMPRESS is not set
CONFIG_CRYPTO_ZSTD=m
CONFIG_ZSTD_COMPRESS=y
CONFIG_ZSTD_DECOMPRESS=y
CONFIG_DECOMPRESS_ZSTD=y

and I think these options:

Code: Select all

CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y

are must to gone. What do you think about this?

BTW you can try F2FS. I will try it too on my test USB flash drive
Porteus with ZSTD kernel boots faster, but the size is huge.
In your opinion, which is better size or boot up speed?

[babam]

and I think these options:

Code: Select all

CONFIG_MODULE_COMPRESS=y
CONFIG_MODULE_COMPRESS_GZIP=y

What's the benefits of that?

BTW you can try F2FS. I will try it too on my test USB flash drive

Don't forget to provide the 'extra_ attr,compression' options at filesystem creation.

Code: Select all

# mkfs.f2fs -O extra_attr,compression /dev/sdXX
# mount -o compress_algorithm=zstd /dev/sdXX /mnt/sdXX

Porteus with ZSTD kernel boots faster, but the size is huge.
In your opinion, which is better size or boot up speed?

Speed, i love speed.

[Blaze]

What's the benefits of that?

No benefit, that's what I'm talking about above.
Grandfather GZIP is activated instead of CONFIG_MODULE_COMPRESS_ZSTD=y

Отправлено спустя 6 минут 40 секунд:

Don't forget to provide the 'extra_ attr,compression' options at filesystem creation.

Info https://wiki.archlinux.org/index.php/F2FS#Compression
not bad if this F2FS mount option add in fstab by default automatically and Gparted able to format with extra_attr,compression options

[roadie]

Blaze,

I'm not sure why your config isn't keeping the changes, I just built a kernel with xz compressed modules with no problems. It boots fine, but there's really very little difference in boot times between the gzip and xz compressed modules. I'll compile a kernel with uncompressed modules and check those times.

What patch are you using for zstd module compression in the kernel?

Code: Select all

stickbox:[guest]:~$ cd /tmp
stickbox:[guest]:/tmp$ ls
000-kernel-5.10.4-aufs-xz.xzm  mc-root	mloop  palemoon_guest  xdg-runtime-guest
stickbox:[guest]:/tmp$ lsxzm 000-kernel-5.10.4-aufs-xz.xzm
/lib/modules
/lib/modules/5.10.4-porteus
/lib/modules/5.10.4-porteus/build
/lib/modules/5.10.4-porteus/kernel
/lib/modules/5.10.4-porteus/kernel/arch
/lib/modules/5.10.4-porteus/kernel/arch/x86
/lib/modules/5.10.4-porteus/kernel/arch/x86/crypto
/lib/modules/5.10.4-porteus/kernel/arch/x86/crypto/aesni-intel.ko.xz
/lib/modules/5.10.4-porteus/kernel/arch/x86/crypto/crc32-pclmul.ko.xz

As for zstd, a patched kernel builds and boots, but the modules are not decompressed because kmod needs to be patched for zstd. When this is done, the kernels can be loaded, but they don't get decompressed at boot because a depmod-a command needs to be run first.....I don't know why this is.

I haven't found anywhere that I can run the depmod command in early boot, and the kernel modules aren't found until that's done. It may be a missing setting in the config that's causing this. I would think that if gzipped and xz compressed modules are found right away, zstd compressed should be as well, but that's not the case.

[roadie]

I just tried a kernel with uncompressed modules, and I really don't see much difference in boot time between uncompressed and xz or gz compressed.....maybe 1 second at best. I don't think it's worth the extra work to make zstd work with compressed modules, at least not for me.......but I probably will.....it's a new shiny thing and I want it.

[Blaze]

What patch are you using for zstd module compression in the kernel?

Damn, it's my fault. Don't laugh at me, but I didn't used the zstd module compression patches
Seems I must to use 0009-add-zstd-compressed-modules.patch

Code: Select all

diff --git a/Makefile b/Makefile
index f900c23b8291..387e83c4373e 100644
--- a/Makefile
+++ b/Makefile
@@ -971,8 +971,8 @@ endif # INSTALL_MOD_STRIP
 export mod_strip_cmd
 
 # CONFIG_MODULE_COMPRESS, if defined, will cause module to be compressed
-# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP
-# or CONFIG_MODULE_COMPRESS_XZ.
+# after they are installed in agreement with CONFIG_MODULE_COMPRESS_GZIP,
+# CONFIG_MODULE_COMPRESS_XZ, or CONFIG_MODULE_COMPRESS_ZSTD.
 
 mod_compress_cmd = true
 ifdef CONFIG_MODULE_COMPRESS
@@ -982,6 +982,9 @@ ifdef CONFIG_MODULE_COMPRESS
   ifdef CONFIG_MODULE_COMPRESS_XZ
     mod_compress_cmd = xz -f
   endif # CONFIG_MODULE_COMPRESS_XZ
+  ifdef CONFIG_MODULE_COMPRESS_ZSTD
+    mod_compress_cmd = zstd -T0 -20 --ultra --rm -f
+  endif
 endif # CONFIG_MODULE_COMPRESS
 export mod_compress_cmd
 
diff --git a/init/Kconfig b/init/Kconfig
index a34064a031a5..345f068d0f62 100644
--- a/init/Kconfig
+++ b/init/Kconfig
@@ -2102,8 +2102,8 @@ config MODULE_COMPRESS
 	bool "Compress modules on installation"
 	help
 
-	  Compresses kernel modules when 'make modules_install' is run; gzip or
-	  xz depending on "Compression algorithm" below.
+	  Compresses kernel modules when 'make modules_install' is run; gzip,
+	  xz, or zstd depending on "Compression algorithm" below.
 
 	  module-init-tools MAY support gzip, and kmod MAY support gzip and xz.
 
@@ -2125,7 +2125,7 @@ choice
 	  This determines which sort of compression will be used during
 	  'make modules_install'.
 
-	  GZIP (default) and XZ are supported.
+	  GZIP (default), XZ, and ZSTD are supported.
 
 config MODULE_COMPRESS_GZIP
 	bool "GZIP"
@@ -2133,6 +2133,9 @@ config MODULE_COMPRESS_GZIP
 config MODULE_COMPRESS_XZ
 	bool "XZ"
 
+config MODULE_COMPRESS_ZSTD
+	bool "ZSTD"
+
 endchoice
 
 config MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS

Отправлено спустя 3 минуты 5 секунд:

but the modules are not decompressed because kmod needs to be patched for zstd. When this is done, the kernels can be loaded, but they don't get decompressed at boot because a depmod-a command needs to be run first.....I don't know why this is.

Code: Select all

module-init-tools MAY support gzip, and kmod MAY support gzip and xz.

in the 0009-add-zstd-compressed-modules.patch

[roadie]

Blaze,

I saw the warnings about kmod in the patch, but had to try anyway.....it's right, standard kmod doesn't work for zstd. I'm wondering if the 0009-add-zstd-compressed-modules.patch is all that's needed, maybe other patches are needed. The patched kmod does work to load the modules, but for some reason it isn't doing it automatically at boot.

I'm compiling another kernel right now with additional patches applied, I'll soon see if it makes a difference.