Porteus Kernel Builder

Here is a place for your projects which are not officially supported by the Porteus Team. For example: your own kernel patched with extra features; desktops not included in the standard ISO like Gnome; base modules that are different than the standard ISO, etc...
fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

Re: Porteus Kernel Builder

Post#121 by fulalas » 14 Jun 2017, 06:55

@neko, I've finished the first version of the scripts for x64. Bare in mind that they require super user (sudo) privileges in order to correctly work. You can put each one of them associated to a unique checkbox in your app -- preferentially call them after everything since they're doing some clean up in the end. Also, I put this line kernelVersion=4.11.4 hardcoded in both scripts because I don't know how you're setting this string. So you need to change kernelVersion=4.11.4 to something dynamic, OK? Finally, I'm considering a initial folder, so you need to check this too.

Both scripts create modules that follow Porteus naming patters, but add kernel version in the end (just before file extension). Example: 000-kernel-4.11.4.xzm crippled_sources-4.11.4.xzm.

Feel free to ask, suggest or change anything you want. :)

Here is the script that creates a kernel module and copy it to Porteus current running unit (usually the user USB stick):

Code: Select all

#set variables
porteusRootDir=`grep -q copy2ram /proc/cmdline && echo "/mnt/live/memory/copy2ram" || grep -A1 "Porteus data found in" /var/log/porteus-livedbg | tail -n1`
kernelVersion=4.11.4

#create file structure
cd v$kernelVersion/64/lib/modules/$kernelVersion-porteus
rm build
ln -s /usr/src/linux-$kernelVersion build
rm source
ln -s /usr/src/linux source
cd ../../..
mkdir kernelModule
cp -r lib/* kernelModule

#if user has kernel files outside a Linux partition then permissions will be wrong so we need to execute the command below
#sudo chown root:root -R kernelModule

#create module
dir2xzm kernelModule 000-kernel-$kernelVersion.xzm

#backup current kernel and vmlinuz
rename xzm xzm_ $porteusRootDir/base/000-*.xzm &>/dev/null
rm $porteusRootDir/../boot/syslinux/vmlinuz_ &>/dev/null
mv $porteusRootDir/../boot/syslinux/vmlinuz $porteusRootDir/../boot/syslinux/vmlinuz_

#copy new kernel and vmlinuz
cp 000-kernel-$kernelVersion.xzm $porteusRootDir/base
cp vmlinuz $porteusRootDir/../boot/syslinux/

#clean up
rm -f 000-kernel-$kernelVersion.xzm
rm -rf kernelModule
And here is the script that creates crippled_sources module and copy it to Porteus current running unit (usually the user USB stick):

Code: Select all

#set variables
porteusRootDir=`grep -q copy2ram /proc/cmdline && echo "/mnt/live/memory/copy2ram" || grep -A1 "Porteus data found in" /var/log/porteus-livedbg | tail -n1`
kernelVersion=4.11.4
cd v$kernelVersion/64/linux-$kernelVersion

#remove useless files
rm vmlinux* &>/dev/null
find . -maxdepth 99 -type f -name ".gitignore" -delete &>/dev/null
find . -maxdepth 99 -type f -name "*.o" -delete &>/dev/null
find . -maxdepth 99 -type f -name "*.bin" -delete &>/dev/null
find . -maxdepth 99 -type f -name "*.elf" -delete &>/dev/null
find . -maxdepth 99 -type f -name "*.xz" -delete &>/dev/null
rm -rf .tmp_versions &>/dev/null
rm -rf Documentation &>/dev/null
rm -rf drivers &>/dev/null
rm -rf firmware &>/dev/null
rm -rf fs &>/dev/null
rm -rf net &>/dev/null
rm -rf sound &>/dev/null
rm arch/x86/boot/bzImage
rm arch/x86/boot/compressed/vmlinux
cp -r arch/x86 .
rm -rf arch
mkdir arch
mv x86 arch

#create file structure
cd ..
mkdir -p crippled_sources/usr/src
mv linux-$kernelVersion crippled_sources/usr/src
cd crippled_sources/usr/src
ln -s /usr/src/linux-$kernelVersion linux
cd ../../..

#create module
dir2xzm crippled_sources crippled_sources-$kernelVersion.xzm

#backup current crippled sources
rename xzm xzm_ $porteusRootDir/base/crippled_sources*.xzm &>/dev/null

#copy new crippled_sources
cp crippled_sources-$kernelVersion.xzm $porteusRootDir/base/

#clean up
rm -f crippled_sources-$kernelVersion.xzm
rm -rf crippled_sources
Last edited by fulalas on 14 Jun 2017, 23:35, edited 2 times in total.

neko
DEV Team
DEV Team
Posts: 2109
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

Re: Porteus Kernel Builder

Post#122 by neko » 14 Jun 2017, 08:04

@fulalas
Excuse me.
I couldn't response about your spec "kernel builder".

Now, I have many issues on introducing "deepin" desktop into "APorteus".
More efforts are needed for me to run the "deepin" in APorteus.

Please give me a little time.

Thanks.

fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

Re: Porteus Kernel Builder

Post#123 by fulalas » 14 Jun 2017, 08:19

@neko, no problem! We don't need to hurry. Take your time. :)

BTW, I've noticed something different with your kernel 4.11.4 booting from VirtualBox. During the boot we have these 2 first messages that are well known, but the third one only happens with kernel 4.11.4:

Code: Select all

Loading vmlinuz......
Loading initrd.xz...ready
[      0.437035]ic2-parport-light: adapter type unspecified
Maybe this is a consequence of your kernel build being so much smaller than Porteus official, so it misses somethings. What you think?

beny
Full of knowledge
Full of knowledge
Posts: 2086
Joined: 02 Jan 2011, 11:33
Location: italy

Re: Porteus Kernel Builder

Post#124 by beny » 14 Jun 2017, 20:29

hi fulalas seem dkms is the answer ,well in arch work like a charm do the upgrade of all the software that is linked to the kernel like nvidia driver or virtualbox.

fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

Re: Porteus Kernel Builder

Post#125 by fulalas » 14 Jun 2017, 23:25

@beny, thanks for your suggestion, but I'm a bit confused. Dkms is the answer to exactly what? I read about it and on Arch Wiki and it seems that is doesn't make much sense since VirtualBox already comes with a kernel module builder, right? Anyway, I tried dkms with VirtualBox and it works, but I didn't see any improvement whatsoever -- it still lacks 3D acceleration and it doesn't create vboxvideo.so . What am I missing here?

beny
Full of knowledge
Full of knowledge
Posts: 2086
Joined: 02 Jan 2011, 11:33
Location: italy

Re: Porteus Kernel Builder

Post#126 by beny » 14 Jun 2017, 23:40

hi in arch if you have installed it the change of kernel version is followed by the upgrade of video driver sat card reader and virtualbox,in arch this step is automatic in porteus or slackware you have to run dkms standalone i think but not shure. you can run the script for recreate the kernel module every way that change /etc/rc.d/init.d/vboxdrv
Dynamic Kernel Module Support (DKMS) is a program/framework that enables generating Linux kernel modules whose sources generally reside outside the kernel source tree. The concept is to have DKMS modules automatically rebuilt when a new kernel is installed.

fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

Re: Porteus Kernel Builder

Post#127 by fulalas » 14 Jun 2017, 23:50

Yeah, I got it. Thanks for the tip :good:

neko
DEV Team
DEV Team
Posts: 2109
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

Re: Porteus Kernel Builder

Post#128 by neko » 16 Jun 2017, 02:22

@fulalas
In order to understand your request about "VirtualBox",
I am installing "VirtualBox" on APorteus.


Thanks.

--------------------------------------------------------------------------------
New "updated kernel Examples" v4.11.5, v4.9.32, v4.4.72, v4.1.41 & v3.18.57 were offered.

neko
DEV Team
DEV Team
Posts: 2109
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

Re: Porteus Kernel Builder

Post#129 by neko » 18 Jun 2017, 11:48

@fulalas
Your spec "Kernel Builder" was uploaded.
I don't know whether it fits exactly with your need or not.
Because I am still under studying the "Virtualbox".

--------------------------------------------------------------------------------
1. "Porteus Kernel Builder" was updated to mkKernel-17.06.18-noarch-1.xzm
Please refer to http://forum.porteus.org/viewtopic.php? ... 779#p52232


2. New "ISO update Example" v4.11.6, v4.9.33, and v4.4.73 were offered.
Please refer to http://forum.porteus.org/viewtopic.php? ... 779#p52232

The kernel of Porteus ISO can be update to version v4.11.6/v4.9.33/v4.4.73 without compiling.


Thanks.

fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

Re: Porteus Kernel Builder

Post#130 by fulalas » 18 Jun 2017, 23:22

@neko, we're doing some progress here! Great! :good:

My considerations regarding your update:

1- the crippled_sources-4.11.6-64bit.xzm module created is too big (102 mb), probably because you didn't use the clean up part of my script. If you take a look at an official Porteus crippled_sources module created by brokenman, you'll notice that it's small (15 mb) because it doesn't come with lots of useless files inside kernel source folder, like Documentations, binaries and many others. Take a look at '#remove useless files' part of my cripples_sources script;
1.1- another crippled sources module (named crippled_src.xzm) is created inside v4.11.6/lib/64, but this module doesn't seem to work at all. Why are you doing that?
2- the 000-kernel module has both 'build' and 'source' symlinks (inside /lib/modules/4.11.6-porteus) broken. In my script I fix it. You may want to take a look;
2.1- kernel module doesn't have the version in its file name as I suggested in my scripts. This can be useful when we fix the next consideration:
3- neither 000-kernel nor crippled_sources are being copied to Porteus running unit, as I suggest in my scripts. The whole idea I had (of course you can disagree) is to make kernel update something really easy (i.e. automatically) for a beginner user;

Thanks!

User avatar
brokenman
Site Admin
Site Admin
Posts: 6105
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v4 all desktops
Location: Brazil

Re: Porteus Kernel Builder

Post#131 by brokenman » 22 Jun 2017, 21:35

Great initiative!! An automatic kernel updater is something I thought about (and started to implement) but never had time to finish. There may even be some commented code in linuxrc.

Neko if you can get deepin running on arch porteus it will motivate me a lot!

For the kernel here are some things to keep in mind.

1) The location of the kernel could be anywhere. Best if users choose where the kernel is and a test is done to confirm.
2) Same for 000-kernel module
3) copy2ram mode must be used else you will be over writing a module that is in use when replacing 000-kernel which could lead to system instability

The best way would be to store downloaded files somewhere and have linuxrc update them before the system kicks in. linuxrc already has this in place for the base modules but not for the kernel. I believe my automated Porteus builder scripts include the 05_devel module.
How do i become super user?
Wear your underpants on the outside and put on a cape.

Evan
Shogun
Shogun
Posts: 466
Joined: 11 Apr 2016, 09:00
Distribution: Distribution: *

Re: Porteus Kernel Builder

Post#132 by Evan » 22 Jun 2017, 23:01

brokenman wrote:Neko if you can get deepin running on arch porteus it will motivate me a lot!.
When i first saw Deepin posted by Neko i thought it looked really nice but having since read about Deepin around the forums there are some concerns that i was going to put to Neko.

Going by users posts its basically felt that the Deepin OS and Desktop project being based in China is more likely to have a backdoor than SystemD with some going as far as saying it's pretty much 100% certain to have backdoors considering the Chinese Governments law that gives them access to all software/hardware based in China.

China and US clash over software backdoor proposals
http://www.bbc.co.uk/news/technology-31729305
Beijing has rejected President Obama's criticism of its plan to make tech companies put backdoors in their software and share their encryption keys if they want to operate in China.
Some have become even more suspicious of the Deepin project as it suddenly decided to maintain it's own kernel last year and it never discloses what they have done or changed.

Are you sure you want to go down that road with the Porteus and risk alienating the project when people online are already getting there panties in a twist over SystemD lack of disclosure?

:unknown:

User avatar
brokenman
Site Admin
Site Admin
Posts: 6105
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v4 all desktops
Location: Brazil

Re: Porteus Kernel Builder

Post#133 by brokenman » 23 Jun 2017, 01:38

Well I had no idea. I've only seen the deepin desktop in a few videos and thought it looked really polished. I haven't taken a look at it yet. It shouldn't be too difficult to confirm that the systemd being used is modified. Is the desktop project open source? I thought I saw a github account for it at some stage. It's not so odd to maintain their own kernel. We do it. Or do you mean they are creating their own kernel?
How do i become super user?
Wear your underpants on the outside and put on a cape.

Evan
Shogun
Shogun
Posts: 466
Joined: 11 Apr 2016, 09:00
Distribution: Distribution: *

Re: Porteus Kernel Builder

Post#134 by Evan » 23 Jun 2017, 04:49

To be honest i don't know any more than i posted above hence why i was going to put this to Neko to see what he thinks.

Like yourself i thought Deepin looked really nice and over the last few days i've been looking at reviews and i keep noticing user comments about the Chinese government requirement for software backdoors so i thought i had better mention it , i'm not trying to join in with any scaremongering that you shouldn't use it but as an air of caution if it's worth the risk?

One user comment that i thought was a really good point was that if its a Chinese government requirement by law then why would Deepin be exempt from that law either now or in the future when their conduct of potentially misusing Linux started years ago with Redflag linux and in recent years with Redstar linux before they introduced the this law?

:unknown:

fulalas
DEV Team
DEV Team
Posts: 2050
Joined: 26 Oct 2016, 15:34
Distribution: Porteus
Location: Brazil

Re: Porteus Kernel Builder

Post#135 by fulalas » 23 Jun 2017, 05:45

brokenman wrote:Neko if you can get deepin running on arch porteus it will motivate me a lot!
He already made it: http://forum.porteus.org/viewtopic.php? ... 225#p55982
brokenman wrote:1) The location of the kernel could be anywhere. Best if users choose where the kernel is and a test is done to confirm.
I agree. Sometimes RAM isn't enough to store something big as Linux kernel, especially after compilation.
brokenman wrote:2) Same for 000-kernel module
Yep. But maybe we're increasing neko app complexity, don't you think? I was thinking about something really simple with less GUI elements than neko current app. Something with a combo showing kernel versions and a button just to update to that chosen version. KISS principle :good:
brokenman wrote:3) copy2ram mode must be used else you will be over writing a module that is in use when replacing 000-kernel which could lead to system instability
You're totally right. I haven't thought about it. Maybe we can do a backup and overwrite it, asking user for restarting Porteus just after. Is it safe?
brokenman wrote:I believe my automated Porteus builder scripts include the 05_devel module.
Could you provide yours scripts?

Another thing: I noticed that your kernel modules are bigger than the ones created by neko app. Do you know why? Also, during boot on VirtualBox I see this message using neko's kernel: http://forum.porteus.org/viewtopic.php? ... 120#p55969

Post Reply