Building Blocks

[Bogomips]

I have a version of Porteus here working with only overlayfs and no aufs if you feel it would help your cause. The commands (in linuxrc) used to get it to boot may help you though.

Looks like it's all going the way of overlayfs. Will be grateful for link to this Porteus version.

It works when installing slackware packages, but will not work with squashfs modules. Well at least I have not been able to bend it to my will.

Could this squashfs solution still be of relevance?

[Jack]

@ brokenman,

I use sbopkg on laptop-mode-tools and then I try it manually but it didn't work. It error out at the readme file and it was there. I download all the files at slackbuilds.org for laptop-mode-tools needed.

I found skype all ready compiled. Here is my question about skype should I use txz2dir or txz2xzm and which is better?

I found when I download the 001-core there was a packages in the list that was util and it should be util-linux. I found this out "n/portmap-6.0-x86_64-1.txz: Removed. This is replaced by the new libtirpc and rpcbind packages." so it needs to be fixed in 001-core or use the remove package. This is what I found so far.

[brokenman]

Bogomips I can give you the linuxrc file so you can see how to get the system booted, but I don't want to upload the ISO. Thanks for the link. The problem is not that the modules don't load on the bottom read only layer. It is that you can't 'activate' a module once your system is up. Since porteus is all about modules, this is not acceptable for me.

I found skype all ready compiled. Here is my question about skype should I use txz2dir or txz2xzm and which is better?

txz2xzm is sufficient here.

[Bogomips]

Brokenman, the linuxrc will do nicely, thanks.

Jack, you can also look for the missing packages in Alien Bob's live system:

Code: Select all

guest@porteus:~$ mkdir z
guest@porteus:~$ sudo mount slackware64-live-xfce-current.iso z
mount: block device /home/guest/slackware64-live-xfce-current.iso is write-protected, mounting read-only

guest@porteus:~$ ls -sh z/liveslak/system/
total 675M
 11M 0000-slackware_boot-current-x86_64.sxz
219M 0020-slackware_min-current-x86_64.sxz
147M 0020-slackware_xapbase-current-x86_64.sxz
283M 0020-slackware_xbase-current-x86_64.sxz
 12M 0020-slackware_xfcebase-current-x86_64.sxz
4.1M 0099-slackware_zzzconf-current-x86_64.sxz

guest@porteus:~$ mkdir x
guest@porteus:~$ sudo mount -t squashfs  z/liveslak/system/0099-slackware_zzzconf-current-x86_64.sxz x
# No Packages
guest@porteus:~$ ls x/var/log
faillog  lastlog
guest@porteus:~$ sudo umount x

guest@porteus:~$ sudo mount -t squashfs  z/liveslak/system/0020-slackware_xfcebase-current-x86_64.sxz x
# 27 items
guest@porteus:~$ du -sh x/var/log/packages/
285K    x/var/log/packages/
guest@porteus:~$ sudo umount x

guest@porteus:~$ sudo mount -t squashfs  z/liveslak/system/0020-slackware_xbase-current-x86_64.sxz x
# 390 items
guest@porteus:~$ du -sh x/var/log/packages/
2.9M    x/var/log/packages/
guest@porteus:~$ sudo umount x

guest@porteus:~$ sudo mount -t squashfs  z/liveslak/system/0020-slackware_xapbase-current-x86_64.sxz x
31 items
guest@porteus:~$ du -sh x/var/log/packages/
625K    x/var/log/packages/
guest@porteus:~$ sudo umount x

guest@porteus:~$ sudo mount -t squashfs  z/liveslak/system/0020-slackware_min-current-x86_64.sxz x    
113 items
guest@porteus:~$ du -sh x/var/log/packages/
1.7M    x/var/log/packages/
guest@porteus:~$ sudo umount x

guest@porteus:~$ sudo mount -t squashfs  z/liveslak/system/0000-slackware_boot-current-x86_64.sxz x
# No packages
guest@porteus:~$ du -sh x/var/log
du: cannot access ‘x/var/log’: No such file or directory
guest@porteus:~$ sudo umount x

guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo umount z

And if they are not there, are they worth having?

[Jack]

@brokenman,

Was there any change in the packages list of 003-mate, 05-devel and 07-printing? I all ready ran the script on them. That a nice script that you made.

[brokenman]

The below code box contains the linuxrc file that I used to get the overlay system up. I stripped most of the superfluous stuff from the script e.g. changes and copy2ram. The result is a live modular slackware OS with persistence in /mnt/live/memory/changes.

Code: Select all

#!/bin/sh
### linuxc script by fanthom <http://www.porteus.org>
### modified for overlayfs by brokenman

export PATH=/bin:/usr/bin:./

# Enable pivot_root in initramfs to let it behave like initrd:
if [ -d /m ]; then
    mount -nt tmpfs tmpfs -o mode=0755 /m
    cp -a ??* /m 2>/dev/null
    exec switch_root /m linuxrc "$@"
fi

mount -nt proc proc /proc
mount -nt sysfs sysfs /sys
echo 0 >/proc/sys/kernel/printk

clear
echo """Starting init script <http://www.porteus.org/>"""

# Functions:
. ./functions

## Let's start!
mount -nt devtmpfs none /dev

[ -e /dev/console ]  || mknod /dev/console c 5 1
[ -e /dev/null ]     || mknod /dev/null c 1 3
[ -e /dev/tty ]      || mknod /dev/tty c 5 0
[ -e /dev/urandom ]  || mknod /dev/urandom c 1 9
[ -e /dev/random ]   || mknod /dev/random c 1 8
[ -e /dev/zero ]     || mknod /dev/zero c 1 5

# Create /etc/fstab and mount devices:
fstab

debug

# Find *.sgn file:
echo $i"searching for $SGN file"
if [ $IP ]; then BOOTDEV=network; SGNDEV=/mnt/nfs
    for x in `lspci | grep 0200: | cut -d: -f3-4 | sed s/:/.*/g | tr a-z A-Z`; do modprobe `grep $x /lib/modules/$(uname -r)/modules.alias | tail -n1 | rev | cut -d" " -f1 | rev` 2>/dev/null; done
    ls /sys/class/net | grep -q eth || { for x in `find /usr/lib/modules/$(uname -r)/kernel/drivers/net -name "*.ko" | sed 's/.ko//g'`; do modprobe `basename $x` 2>/dev/null; ls /sys/class/net | grep -q eth && break || modprobe -r `basename $x` 2>/dev/null; done; }
    mkdir -p /mnt/nfs/$FOLDER /mnt/nfs/storage; udhcpc; modprobe nfsv4; mount -t nfs4 $IP:/srv/pxe/$FOLDER /mnt/nfs/$FOLDER -o ro,nolock 2>/dev/null || { modprobe nfsv3; mount -t nfs $IP:/srv/pxe/$FOLDER /mnt/nfs/$FOLDER -o ro,nolock 2>/dev/null; }
    MAC=`ifconfig | grep eth0 | cut -d: -f5-7 | sed s/://g | cut -d" " -f1`
    if [ "$CHANGES" = /srv/pxe/storage ]; then
	if lsmod | grep -q nfsv3; then
	    mount -t nfs $IP:/srv/pxe/storage /mnt/nfs/storage -o rw,nolock 2>/dev/null && { mkdir -p /mnt/nfs/storage/client-$MAC/changes/home; CHANGES="/storage/client-$MAC"; }
	else
	    mount -t nfs4 $IP:/srv/pxe/storage /mnt/nfs/storage -o rw,nolock 2>/dev/null && { mkdir -p /mnt/nfs/storage/client-$MAC/changes/home; CHANGES="/storage/client-$MAC"; }
	fi
    fi
elif [ $ISO ]; then SGNDEV=/mnt/isoloop
    locate -e $FROM && { BOOTDEV=/mnt/$DEV; mkdir /mnt/isoloop; mount -o loop /mnt/$DEV/$LPTH /mnt/isoloop; }
else
    if [ $FROM ]; then
	locate -e $FROM/$FOLDER/$SGN
	if [ $? -eq 0 ]; then
	    DIR=`echo $LPTH | rev | cut -d/ -f3- | rev`; [ $DIR ] && FOLDER=$DIR/$FOLDER
	else
	    echo $i"from= cheatcode is incorrect, press enter to search through all devices"
	    read -s; search -e $FOLDER/$SGN
	fi
    else
	search -e $FOLDER/$SGN || lazy -e $FOLDER/$SGN
    fi
    SGNDEV=/mnt/$DEV
fi

[ -e $SGNDEV/$FOLDER/$SGN ] && PTH=$SGNDEV/$FOLDER || . fatal
MODDIR=$PTH/modules
BASDIR=${PTH%/*}

echo $i"using Porteus data from $PTH"

debug

# Make all drivers available:
mount -o loop $PTH/base/000-kernel.xzm /opt/000-kernel 2>/dev/null
mount -o bind /opt/000-kernel/usr/lib/modules /usr/lib/modules 2>/dev/null

# Mount a tmpfs for write access
mount -t tmpfs tmpfs /memory
mkdir -p /memory/changes
mkdir -p /memory/work
mkdir -p /memory/copy2ram
mkdir -p /memory/images

if [ $? -ne 0 ]; then echo -e """cant setup union (aufs) - read only filesystem?\nWhen you finish debugging press Ctrl+Alt+Del to reboot."""; sh; fi

# Find modules:
find $PTH/base $PTH/modules -name "*.xzm" 2>/dev/null | egrep -v "$NOLOAD" | sort >/tmp/modules
find $PTH/optional -name "*.xzm" 2>/dev/null | egrep "$LOAD" | sort >>/tmp/modules

# Populate aufs with modules:
umount /lib/modules /opt/000-kernel 2>/dev/null

echo $i"Mounting modules"
for x in `cat /tmp/modules`; do
    NAME=`basename $x`; mkdir /memory/images/$NAME; mount -o loop $x /memory/images/$NAME 2>/dev/null
    if [ $? -eq 0 ]; then
	echo "  $m  $NAME"
    else
	echo $i"""Cannot read $NAME - corrupted module?"""; rmdir /memory/images/$NAME
    fi
done
echo $i"Bind mounting lib/modules"
mount -no bind /merge/lib/modules /lib/modules 2>/dev/null
debug

echo $i"Creating list of modules OARGS"
# Create a list of modules to be overlayed
for a in $(ls /memory/images/); do a="/memory/images/$a:"; ARGS=$ARGS$a; done
ARGS=`echo $ARGS |sed 's/\(.*\)./\1/'`
debug

# Create an extra empty but mounted writable directory in the overlay
mkdir /memory/images/test
ARGS="$ARGS:/memory/images/test"

echo $i"Creating overlay mounts"
mount -t overlay overlay -o lowerdir=$ARGS,upperdir=/memory/changes,workdir=/memory/work /merge
#mount -t overlay overlay -o lowerdir=$ARGS,upperdir=/memory/changes,workdir=/memory/work /merge

fstab
debug

# Create 7 free loop devices for truecrypt, etc...
echo $i"Creating 7 extra loop devices"
x=`losetup | tail -n1 | cut -d: -f1 | sed 's@/dev/loop@@'`; let y=x+7
while [ $x -le $y ]; do [ -b /dev/loop$y ] && break || mknod /dev/loop$y b 7 $y; let y=y-1; done

cp -af /dev/console /merge/dev
fstab

debug

# Create debug file:
[ -e /tmp/devices ] && { echo "# Recognized devices:" >$livedbg; cat /tmp/devices >>$livedbg; }
[ $BOOTDEV ] && SGNDEV=$BOOTDEV
echo -e "\n# Booting device:\n$SGNDEV\n\n# Porteus data found in:\n$PTH\n\n# Changes are stored in:\n$CHANGES\n\n# Non standard /rootcopy dir:\n$ROOTCOPY\n\n# Modules activated during boot time:" >>$livedbg; cat /tmp/modules >>$livedbg
cp -af $livedbg /merge/var/log/porteus-livedbg

echo $i"changing root directory"
if param noauto; then
    for x in `grep /mnt/ /etc/fstab | cut -d/ -f3`; do
		mkdir -p /merge/mnt/$x
		umount -n /mnt/$x 2>/dev/null && rmdir /mnt/$x
		sed -i '/mnt\/'$x'/d' /etc/fstab ### TESTING
	done
else
    grep /mnt/ /etc/fstab >> /merge/etc/mtab
    for x in `grep /mnt/ /etc/fstab | cut -d/ -f3`; do mkdir -p /merge/mnt/$x; mount -n --move /mnt/$x /merge/mnt/$x; rmdir /mnt/$x; done
fi
sed -i 's/ ntfs / ntfs-3g /g' /etc/fstab
cp -f /etc/fstab /merge/etc 2>/dev/null
umount -n /lib/modules 2>/dev/null
rm -r /lib/* /usr/*

debug
echo """live system is ready now - starting Porteus"""

mkdir /merge/mnt/live
if [ $? -eq 0 ]; then
    pivot_root /merge  /merge/mnt/live
    exec bin/chroot . /usr/bin/init "$@" <dev/console >dev/console 2>&1
else
    echo -e """!!ERROR!!\nSomething went wrong and I cannot continue.\nPress Ctrl+Alt+Del to reboot."""
    sh
fi

Jack there were substantial changes in 05-devel from slackware 14.1 to 14.2 (slackware-current). The most important being everything is compiled against a higher version of gcc. The script makes life easier, no more having to search the slackware-current repo when updating. I am not sure about 07 and 003 as I haven't looked at them yet.

[Jack]

Jack there were substantial changes in 05-devel from slackware 14.1 to 14.2 (slackware-current). The most important being everything is compiled against a higher version of gcc. The script makes life easier, no more having to search the slackware-current repo when updating. I am not sure about 07 and 003 as I haven't looked at them yet.

So on 05-devel I just can't upgrade them to 14.2 (slackware-current)? Cause that what wanted to do.

[brokenman]

Assuming you are working from Porteus-v3.1 (slackware 14.1) the first step would be to create a bootable system with a slackware 14.2 001-core.xzm module. From there you can begin to create your 002-xorg.xzm module. You can pull any package from slackware-current. You shouldn't need to compile anything in order to get a working 001-core module. Once you have a slackware 14.2 system you can boot into (or chroot into) it, then you can update 05-devel using the script and start compiling. So in short, anything that is compiled that you want to use in your slackware 14.2 porteus should be compiled ON slackware 14.2 porteus.

Here is a handy command to get the file ready for the script.

Code: Select all

ls -1 /mnt/live/memory/images/001-core.xzm/var/log/packages > /tmp/001-pkg.txt
for a in $(cat /tmp/01-pkg.txt); do echo ${a%-*-*-*} >> /tmp/001-name.txt; done

[Bogomips]

The problem is not that the modules don't load on the bottom read only layer. It is that you can't 'activate' a module once your system is up. Since porteus is all about modules, this is not acceptable for me.

Next on the agenda (following categorical statement): Standalone aufs module for 4.1.15

Progress so Far

Reached 'Setup aufs:' stage, at which point require to merge brokenman's overlayfs variant.

Relevant diagnostic output:

Code: Select all

Loop and Squashfs Activated
Make all drivers available:
:loop /mnt/sda9/porteus/base/000-kernel.xzm /opt/000-kernel

/opt/000-kernel/
lib
/opt/000-kernel/lib/
cpp             dhcpcd          firmware        modprobe.d      modules
udev

:bind /opt/000-kernel/lib/modules /lib/modules
/lib
ld-uClibc-0.9.33.2.so   ld-uClibc.so.0          libblkid.so.1
libblkid.so.1.1.0       libc.so.0               libcom_err.so.2
libcom_err.so.2.1       libe2p.so.2             libe2p.so.2.3
libext2fs.so.2          libext2fs.so.2.4        libgcc_s.so
libgcc_s.so.1           libpthread-0.9.33.2.so  libpthread.so.0
libuClibc-0.9.33.2.so   libuuid.so.1            libuuid.so.1.3.0
modules

/lib/modules/
4.1.15
/lib/modules/4.1.15/
build                   kernel                  modules.alias
modules.alias.bin       modules.builtin         modules.builtin.bin
modules.dep             modules.dep.bin         modules.devname
modules.order           modules.softdep         modules.symbols
modules.symbols.bin     source
/lib/modules/4.1.15/kernel/
arch            crypto          drivers         fs              kernel
lib             mm              net             security        sound

/lib/modules/4.1.15/kernel/lib/
asn1_decoder.ko         bch.ko                  cordic.ko
crc-ccitt.ko            crc7.ko                 crc8.ko
lru_cache.ko            lz4                     mpi
reed_solomon            test-hexdump.ko         test_bpf.ko
test_firmware.ko        test_module.ko          test_user_copy.ko
ts_bm.ko                ts_fsm.ko               ts_kmp.ko

/lib/modules/4.1.15/kernel/fs/
9p              adfs            affs            afs             autofs4
befs            bfs             binfmt_misc.ko  btrfs           cachefiles
ceph            cifs            coda            cramfs          dlm
ecryptfs        efivarfs        efs             exofs           ext2
ext3            ext4            f2fs            fat             freevxfs
fscache         fuse            gfs2            hfs             hfsplus
hpfs            isofs           jbd             jbd2            jffs2
jfs             logfs           mbcache.ko      minix           ncpfs
nfsd            nilfs2          nls             ntfs            ocfs2
omfs            overlayfs       pstore          qnx4            qnx6
quota           reiserfs        squashfs        sysv            ubifs
udf             ufs             xfs

Create symlinks for porteus scripts:
 -sf /mnt/sda9/porteus/modules  /porteus/modules
 -sf /mnt/sda9/porteus/optional /porteus/optional

Symlinks Done. Setup Changes:

/porteus/
modules
optional

/porteus/optional/
/porteus/modules/

Changes processing finished: mkdir -p /memory/changes/mnt/live
Setup aufs:

Not sure if more has to be added or something taken out

One thing else I noticed with fs processing was that when there wasn't an ext2 to modprobe, ext4 got modprobed instead, and this then worked for the ext2 partition.

guest@porteus:~$ md5sum initrd_250116.img
98f90e789c3e45932eb19804cd28d774 initrd_250116.img

[wread]

This is the best topic I have ever read! I will print it; the title of this little book will be "Thanks Jay!"

I spent many hours doing those things manually! I learned them mostly the hard way....

Regards

[Jack]

I'm still looking for a few more packages. But when I do try and build 001-core should I boot up in text mode or Graphics mode (MATE)? I want and hope it works. I will supply a list of packages I can't fine.

[brokenman]

I spent many hours doing those things manually! I learned them mostly the hard way....

Then we could use your experience here definitely. You may also like to reply to my request for 003-kde packages list. It would help the slackware cause. Thanks.

Jack most probably ... it won't boot the first time. Be prepared for that. Boot into text mode and issue: startx 2> /tmp/startx.log
When it fails you can read the log.

Bogomips it looks good to me. When I get a chance i will upload the overlayfs porteus somewhere in the interest of experimentation. It may be a while though. It can't be called Porteus as it is mostly a lightweight slackware live OS in which you can install slackware packages.

Go forth and conquer.

[Jack]

@ brokenman,

The following packages were not found. But I found most. * - needs to be converted or compiled. They might be out dated.

httpfs2-------------------------httpfs2-0.1.4-x86_64-1dj.txz
ifenslave-----------------------ifenslave-1.1.0.x86.64-pkg-8.txz
iputils---------------------------iputils-20140519.fad11dc.1.x86.64-pkg-1.txz
laptop-mode-tools----------laptop-mode-tools-1.61-x86_64-1_SBo.tgz
p7zip---------------------------p7zip-15.09-x86_64-1alien.tgz
pptp----------------------------pptp-1.8.0-x86_64-1_SBo.tgz
sshfs-fuse---------------------sshfs-fuse-2.5-x86_64-1alien.tgz
tofrodos-----------------*-----tofrodos-1.7.13.tar.gz
unrar---------------------------unrar-4.2.4-x86_64-1alien.tgz
usm----------------------------usm-3.1.9-noarch-1.txz
webfs--------------------------webfs-1.21.x86.64-pkg-11.txz
wgetpaste--------------------wgetpaste-2.25-noarch-2_SBo.tgz

This was missing in the list brokenman has posted. Can I use an unstripped version?

kernel-firmware-stripped

Do any need to be change?

[brokenman]

These packages are not essential to get a bootable system however the sources you have chosen are fine.

Can I use an unstripped version?

Yes, just the size will be bigger. That's all. The kernel firmware has been moved into 000-kernel and is no longer stripped.

Do any need to be change?

No

[Jack]

Why I added those files you had them is the list I downloaded. If I new the minimum amount of packages that all I would use but I don't know for sure.