Porteus Kernel Builder

[neko]

Most recent version of build examples was updated.
Kernel Version: 5.5-rc4

Note: AUFS patch "5.5" is not yet ready, and
AUFS patch "5.x-rcN-20191223" is not yet ready for kernel version 5.5-rc4.
[Patching error]
1 out of 1 hunk FAILED -- saving rejects to file security/device_cgroup.c.rej
[Compile error] On usage of "rwsem_release()", the number of argument is mismatch.
fs/aufs/i_op.c:
fs/aufs/wkq.c:

These (device_cgroup.c, i_op.c, wkq.c) were patched by manual.
Porteus Kernel Builder (Post by neko #75134)


Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[neko]

Most recent version of build examples was updated.
Kernel Version: 5.4.7, 4.19.92


Note: AUFS patch "5.4" ("5.4-20191223") has been ready.
Apply all patches (basic patches and optional patch).


Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[Blaze]

Happy New Year neko

My 1st request in 2020:

Code: Select all

# the actual:
# CONFIG_IPV6_MULTIPLE_TABLES is not set
# should be changed to:
CONFIG_IPV6_MULTIPLE_TABLES=y

https://www.linuxquestions.org/question ... ost6074253

[neko]

@Blaze
Happy New Year!!

update the config of kernel 5.5-rc4

Code: Select all

# diff .config.old .config
966c966,967
< # CONFIG_IPV6_MULTIPLE_TABLES is not set
---
> CONFIG_IPV6_MULTIPLE_TABLES=y
> # CONFIG_IPV6_SUBTREES is not set
#

32bit-ALL-kernel5.5-rc4-2.tar (90 M)
http://www.mediafire.com/file/lhqqtvyof ... -rc4-2.tar
md5sum: f46744dbb86f376cdfa103533dfa9eb3 32bit-ALL-kernel5.5-rc4-2.tar

64bit-ALL-kernel5.5-rc4-2.tar (95 M)
http://www.mediafire.com/file/an97x4vuq ... -rc4-2.tar
md5sum: 6400b2b6d2c55fd62637b74877b66394 64bit-ALL-kernel5.5-rc4-2.tar


Thanks.

[neko]

1. "Porteus Kernel Builder" was updated.
Version: 20.01.05

Update the config of kernel 5.5-rc4, 5.4.8, 4.19.93
(CONFIG_IPV6_MULTIPLE_TABLES=y)


2. Most recent version of build examples was updated.
Kernel Version: 5.4.8, 4.19.93

Note: update the config of kernel 5.5-rc4, 5.4.8, 4.19.93

Code: Select all

# diff .config.old .config
966c966,967
< # CONFIG_IPV6_MULTIPLE_TABLES is not set
---
> CONFIG_IPV6_MULTIPLE_TABLES=y
> # CONFIG_IPV6_SUBTREES is not set
#
# cd /v5.4.8/64/linux-5.4.8
# diff .config.old .config
3c3
< # Linux/x86 5.4.3 Kernel Configuration
---
> # Linux/x86 5.4.8 Kernel Configuration
959c959,960
< # CONFIG_IPV6_MULTIPLE_TABLES is not set
---
> CONFIG_IPV6_MULTIPLE_TABLES=y
> # CONFIG_IPV6_SUBTREES is not set
#
# cd /v4.19.93/64/linux-4.19.93
# diff .config.old .config
3c3
< # Linux/x86 4.19.89 Kernel Configuration
---
> # Linux/x86 4.19.93 Kernel Configuration
975c975,976
< # CONFIG_IPV6_MULTIPLE_TABLES is not set
---
> CONFIG_IPV6_MULTIPLE_TABLES=y
> # CONFIG_IPV6_SUBTREES is not set
#

Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[neko]

Most recent version of build examples was updated.
Kernel Version: 5.5-rc5


Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[fulalas]

@neko, what compiler flags and kernel flags are you using currently? I've just read a Phoronix article about compiling optimizations and maybe you want to try somethings to see if it improves boot time. I can help you with the tests using kernel 5.4.x!

Here is my initial suggestion based on the article I mentioned above (you need to add it before calling any make):

Code: Select all

export CXXFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common -fvisibility-inlines-hidden"

export CFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common"

export FFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition"

export KBUILD_KCONFIG="--build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v"

[neko]

1.@fulalas
"what compiler flags and kernel flags are you using currently?"
--->
A kernel is built as followed.
"make -j4"

[compiler flags]
-j4: build performance up on my PC.

No other flags than ones of build paramaters that are included in kernel source.

Excuse me for not understanding the article well.
If you have a time,
you try building kernel, upload the result, and tell us the procedure of kernel build for Porteus.
You can find all procedure of "Kernel Builder" as shell script.
Please refer to the top article of this thread.
Porteus Kernel Builder

Thanks.


2. Most recent version of build examples was updated.
Kernel Version: 5.4.10, 4.19.94

Note:
Kernel version 5.4.9 has been skipped.

Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[fulalas]

@neko, I'm trying but I have some questions.

1- I noticed that your application has /usr/local/share/mkKernel/inc/Compile.inc and /usr/local/sbin/mkKernel, and both scripts call make. Why?
2- when it asks for GCC plugins, what you usually answer? If I try 'yes' then comes a series of questions that I'm not sure how to answer;
3- right now I trying without GCC plugins and adding this in your Compile.inc script before calling any make:

Code: Select all

export CXXFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common -fvisibility-inlines-hidden"

export CFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common"

export FFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition"

export KBUILD_KCONFIG="--build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v"

I'll let you know the results as soon as I finish it

[fulalas]

So I managed to compile it successfully, but the boot time and memory consumption didn't change. I wonder if we should keep these flags as the default anyway, because maybe my result is specific to my machine, which means other users may experience a better result

Do you wanna try?

[neko]

@fulalas

1- I noticed that your application has /usr/local/share/mkKernel/inc/Compile.inc and /usr/local/sbin/mkKernel, and both scripts call make. Why?

[/usr/local/sbin/mkKernel]
This is a CUI (character user interface) command.
It is old version. Now it is not maintenanced.

[/usr/local/share/mkKernel/inc/Compile.inc]
This shell script is dynamically included by GUI /usr/local/sbin/guimkKernel with the dot '.' command.

guimkKernel uses '.inc' files in /usr/local/share/mkKernel/inc/.
guimkKernel creates the shell script which will do the jobs that user selected via GUI window.
The created shell script (/tmp/work$$/perform) is performed by guimkKernel.
The created shell script uses commands (shell scripts) in /usr/local/share/mkKernel/bin/.

Therefore the really used shell script to build kernel is this created shell script (/tmp/work$$/perform).
You can get this created shell script by modifying guimkKernel.
This modifying guimkKernel procedure is explained at "[Detail Explanation]" in the top article of this thread.
-------------------------------------------
Action: Compile
by checking on "compile" item.

Code: Select all

###=== Compile ===###
cd /home/guest/porteus/kernel/v4.9.167/64/
rm -fr vmlinuz lib
cd linux-4.9.167
make oldconfig
make -j4
cp arch/x86/boot/bzImage ../vmlinuz
make modules_install INSTALL_MOD_PATH=../
make firmware_install INSTALL_MOD_PATH=../ 2>/dev/null
cp .config ../64bit.config
cd ../
if [ ! -f vmlinuz -o ! -d lib ]
then
	echo "compile error" >> /tmp/work951/errorMSG
	exit 1
fi
DIR=`/bin/ls lib/modules/`
rm lib/modules/$DIR/build lib/modules/$DIR/source
ln -sf /usr/src/linux lib/modules/$DIR/build
ln -sf /usr/src/linux lib/modules/$DIR/source

-------------------------------------------

[How to change "make parameters" of "Kernel Builder"]
1. Get a created shell script and then modify it directly
or
2. Modify the "make" creating routine of the function "DoCompile()" of the file "/usr/local/share/mkKernel/inc/Compile.inc".

Code: Select all

#======================================
# DoCompile
#--------------------------------------
DoCompile()
{
if [ $EXCOMP = false ]
then
	return 0
fi
cat >> $WORK/perform << EOF
###=== Compile ===###
cd $KDIR/v$KVER/$COMPARCH/
rm -fr vmlinuz lib
cd linux-${KVER}
make oldconfig
CPU_THREADS=\`nproc --all\`
make -j\$CPU_THREADS
cp arch/x86/boot/bzImage ../vmlinuz
make modules_install INSTALL_MOD_PATH=../
make firmware_install INSTALL_MOD_PATH=../ 2>/dev/null
cp .config ../${COMPARCH}bit.config
cd ../
if [ ! -f vmlinuz -o ! -d lib ]
then
	echo "compile error" >> $WORK/errorMSG
	exit 1
fi
DIR=\`/bin/ls lib/modules/\`
rm lib/modules/\$DIR/build lib/modules/\$DIR/source
ln -sf /usr/src/linux lib/modules/\$DIR/build
ln -sf /usr/src/linux lib/modules/\$DIR/source

EOF

return 0
}

Caution
Please distinguish shell instructions for "guimkKernel" and ones for "created shell script".
example:
cd $KDIR/v$KVER/$COMPARCH/ : shell instructions for "guimkKernel"
CPU_THREADS=\`nproc --all\` : shell instructions for "created shell script"


2- when it asks for GCC plugins, what you usually answer? If I try 'yes' then comes a series of questions that I'm not sure how to answer;
I don't know "GCC plugin" functions well.
https://gcc.gnu.org/wiki/plugins
But, after build start, if it might need user's interactive response,
it might be difficult for "Kernel Builder" to use "GCC plugin" functions.


3- right now I trying without GCC plugins and adding this in your Compile.inc script before calling any make:

Code: Select all

#======================================
# DoCompile
#--------------------------------------
DoCompile()
{
if [ $EXCOMP = false ]
then
	return 0
fi
cat >> $WORK/perform << EOF
###=== Compile ===###
cd $KDIR/v$KVER/$COMPARCH/
rm -fr vmlinuz lib
cd linux-${KVER}
make oldconfig

## patch by fulalas 2020.01.12 ##
export CXXFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common -fvisibility-inlines-hidden"
export CFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common"
export FFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition"
export KBUILD_KCONFIG="--build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v"
## end of patch ##

CPU_THREADS=\`nproc --all\`
make -j\$CPU_THREADS
cp arch/x86/boot/bzImage ../vmlinuz
make modules_install INSTALL_MOD_PATH=../
make firmware_install INSTALL_MOD_PATH=../ 2>/dev/null
cp .config ../${COMPARCH}bit.config
cd ../
if [ ! -f vmlinuz -o ! -d lib ]
then
	echo "compile error" >> $WORK/errorMSG
	exit 1
fi
DIR=\`/bin/ls lib/modules/\`
rm lib/modules/\$DIR/build lib/modules/\$DIR/source
ln -sf /usr/src/linux lib/modules/\$DIR/build
ln -sf /usr/src/linux lib/modules/\$DIR/source

EOF

return 0
}

"Do you wanna try?"
---->
I will try the above DoCompile() that includes your patch.

Would you tell me how to change KBUILD_KCONFIG for 32bit kernel ?



---------------------------------------------------------------
The created shell script by guimkKernel that includes your patch.

Code: Select all

#!/bin/sh
###=== Compile ===###
cd /home/guest/kernel/v5.5-rc5/64/
rm -fr vmlinuz lib
cd linux-5.5-rc5
make oldconfig

## patch by fulalas 2020.01.12 ##
export CXXFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common -fvisibility-inlines-hidden"
export CFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wformat -Wformat-security -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -fno-semantic-interposition -ffat-lto-objects -fno-signed-zeros -fno-trapping-math -fassociative-math -Wl,-sort-common"
export FFLAGS="-g -O3 -feliminate-unused-debug-types -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=32 -Wl,--copy-dt-needed-entries -m64 -fasynchronous-unwind-tables -Wp,-D_REENTRANT -ftree-loop-distribute-patterns -Wl,-z -Wl,now -Wl,-z -Wl,relro -malign-data=abi -fno-semantic-interposition"
export KBUILD_KCONFIG="--build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -v"
## end of patch ##

CPU_THREADS=`nproc --all`
make -j$CPU_THREADS
cp arch/x86/boot/bzImage ../vmlinuz
make modules_install INSTALL_MOD_PATH=../
make firmware_install INSTALL_MOD_PATH=../ 2>/dev/null
cp .config ../64bit.config
cd ../
if [ ! -f vmlinuz -o ! -d lib ]
then
	echo "compile error" >> /tmp/work95953/errorMSG
	exit 1
fi
DIR=`/bin/ls lib/modules/`
rm lib/modules/$DIR/build lib/modules/$DIR/source
ln -sf /usr/src/linux lib/modules/$DIR/build
ln -sf /usr/src/linux lib/modules/$DIR/source

=== RESULT ===
64bit-ALL-kernel5.5-rc5-2.tar (95 M)
http://www.mediafire.com/file/g3ot2sjwg ... -rc5-2.tar
md5sum: a2fa2a8cf057209516bf62f60ba8841a 64bit-ALL-kernel5.5-rc5-2.tar



Thanks.

[neko]

Most recent version of build examples was updated.
Kernel Version: 5.5-rc6, 5.4.11, 4.19.95

Note: AUFS patch "5.5" is not yet ready, but
AUFS patch "5.x-rcN-20200113" is ready for kernel version 5.5-rc6.


Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[fulalas]

@neko, thanks for your detailed answer. I asked in LinuxQuestion forum somethings that are still not clear: https://www.linuxquestions.org/question ... ost6078000

Since the boot time hasn't changed a bit, either we're doing something wrong or Porteus bottleneck is not in kernel.

[neko]

Most recent version of build examples was updated.
Kernel Version: 5.4.12, 4.19.96



Please refer to the top article of this thread.
Porteus Kernel Builder


Thanks.

[babam]

@fulalas

Try this kernel config

Code: Select all

CONFIG_CPU_FREQ_DEFAULT_GOV_PERFORMANCE=y
# CONFIG_CPU_FREQ_DEFAULT_GOV_ONDEMAND is not set