Page 3 of 9
Re: Building Blocks
Posted: 23 Jan 2016, 01:07
by Bogomips
brokenman wrote:I have a version of Porteus here working with only overlayfs and no aufs if you feel it would help your cause. The commands (in linuxrc) used to get it to boot may help you though.
Looks like it's all going the way of overlayfs. Will be grateful for link to this Porteus version.
brokenman wrote: It works when installing slackware packages, but will not work with squashfs modules. Well at least I have not been able to bend it to my will.
Could this
squashfs solution still be of relevance?
Re: Building Blocks
Posted: 23 Jan 2016, 01:58
by Jack
@ brokenman,
I use sbopkg on laptop-mode-tools and then I try it manually but it didn't work. It error out at the readme file and it was there. I download all the files at slackbuilds.org for laptop-mode-tools needed.
I found skype all ready compiled. Here is my question about skype should I use txz2dir or txz2xzm and which is better?
I found when I download the 001-core there was a packages in the list that was util and it should be util-linux. I found this out "n/portmap-6.0-x86_64-1.txz: Removed. This is replaced by the new libtirpc and rpcbind packages." so it needs to be fixed in 001-core or use the remove package. This is what I found so far.
Re: Building Blocks
Posted: 23 Jan 2016, 02:48
by brokenman
Bogomips I can give you the linuxrc file so you can see how to get the system booted, but I don't want to upload the ISO. Thanks for the link. The problem is not that the modules don't load on the bottom read only layer. It is that you can't 'activate' a module once your system is up. Since porteus is all about modules, this is not acceptable for me.
I found skype all ready compiled. Here is my question about skype should I use txz2dir or txz2xzm and which is better?
txz2xzm is sufficient here.
Re: Building Blocks
Posted: 23 Jan 2016, 15:32
by Bogomips
Brokenman, the linuxrc will do nicely, thanks.
Jack, you can also look for the missing packages in Alien Bob's live system:
Code: Select all
guest@porteus:~$ mkdir z
guest@porteus:~$ sudo mount slackware64-live-xfce-current.iso z
mount: block device /home/guest/slackware64-live-xfce-current.iso is write-protected, mounting read-only
guest@porteus:~$ ls -sh z/liveslak/system/
total 675M
11M 0000-slackware_boot-current-x86_64.sxz
219M 0020-slackware_min-current-x86_64.sxz
147M 0020-slackware_xapbase-current-x86_64.sxz
283M 0020-slackware_xbase-current-x86_64.sxz
12M 0020-slackware_xfcebase-current-x86_64.sxz
4.1M 0099-slackware_zzzconf-current-x86_64.sxz
guest@porteus:~$ mkdir x
guest@porteus:~$ sudo mount -t squashfs z/liveslak/system/0099-slackware_zzzconf-current-x86_64.sxz x
# No Packages
guest@porteus:~$ ls x/var/log
faillog lastlog
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo mount -t squashfs z/liveslak/system/0020-slackware_xfcebase-current-x86_64.sxz x
# 27 items
guest@porteus:~$ du -sh x/var/log/packages/
285K x/var/log/packages/
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo mount -t squashfs z/liveslak/system/0020-slackware_xbase-current-x86_64.sxz x
# 390 items
guest@porteus:~$ du -sh x/var/log/packages/
2.9M x/var/log/packages/
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo mount -t squashfs z/liveslak/system/0020-slackware_xapbase-current-x86_64.sxz x
31 items
guest@porteus:~$ du -sh x/var/log/packages/
625K x/var/log/packages/
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo mount -t squashfs z/liveslak/system/0020-slackware_min-current-x86_64.sxz x
113 items
guest@porteus:~$ du -sh x/var/log/packages/
1.7M x/var/log/packages/
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo mount -t squashfs z/liveslak/system/0000-slackware_boot-current-x86_64.sxz x
# No packages
guest@porteus:~$ du -sh x/var/log
du: cannot access ‘x/var/log’: No such file or directory
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo umount x
guest@porteus:~$ sudo umount z
And if they are not there, are they worth having?
Re: Building Blocks
Posted: 23 Jan 2016, 17:31
by Jack
@brokenman,
Was there any change in the packages list of 003-mate, 05-devel and 07-printing? I all ready ran the script on them. That a nice script that you made.
Re: Building Blocks
Posted: 23 Jan 2016, 21:36
by brokenman
The below code box contains the linuxrc file that I used to get the overlay system up. I stripped most of the superfluous stuff from the script e.g. changes and copy2ram. The result is a live modular slackware OS with persistence in /mnt/live/memory/changes.
Code: Select all
#!/bin/sh
### linuxc script by fanthom <http://www.porteus.org>
### modified for overlayfs by brokenman
export PATH=/bin:/usr/bin:./
# Enable pivot_root in initramfs to let it behave like initrd:
if [ -d /m ]; then
mount -nt tmpfs tmpfs -o mode=0755 /m
cp -a ??* /m 2>/dev/null
exec switch_root /m linuxrc "$@"
fi
mount -nt proc proc /proc
mount -nt sysfs sysfs /sys
echo 0 >/proc/sys/kernel/printk
clear
echo """Starting init script <http://www.porteus.org/>"""
# Functions:
. ./functions
## Let's start!
mount -nt devtmpfs none /dev
[ -e /dev/console ] || mknod /dev/console c 5 1
[ -e /dev/null ] || mknod /dev/null c 1 3
[ -e /dev/tty ] || mknod /dev/tty c 5 0
[ -e /dev/urandom ] || mknod /dev/urandom c 1 9
[ -e /dev/random ] || mknod /dev/random c 1 8
[ -e /dev/zero ] || mknod /dev/zero c 1 5
# Create /etc/fstab and mount devices:
fstab
debug
# Find *.sgn file:
echo $i"searching for $SGN file"
if [ $IP ]; then BOOTDEV=network; SGNDEV=/mnt/nfs
for x in `lspci | grep 0200: | cut -d: -f3-4 | sed s/:/.*/g | tr a-z A-Z`; do modprobe `grep $x /lib/modules/$(uname -r)/modules.alias | tail -n1 | rev | cut -d" " -f1 | rev` 2>/dev/null; done
ls /sys/class/net | grep -q eth || { for x in `find /usr/lib/modules/$(uname -r)/kernel/drivers/net -name "*.ko" | sed 's/.ko//g'`; do modprobe `basename $x` 2>/dev/null; ls /sys/class/net | grep -q eth && break || modprobe -r `basename $x` 2>/dev/null; done; }
mkdir -p /mnt/nfs/$FOLDER /mnt/nfs/storage; udhcpc; modprobe nfsv4; mount -t nfs4 $IP:/srv/pxe/$FOLDER /mnt/nfs/$FOLDER -o ro,nolock 2>/dev/null || { modprobe nfsv3; mount -t nfs $IP:/srv/pxe/$FOLDER /mnt/nfs/$FOLDER -o ro,nolock 2>/dev/null; }
MAC=`ifconfig | grep eth0 | cut -d: -f5-7 | sed s/://g | cut -d" " -f1`
if [ "$CHANGES" = /srv/pxe/storage ]; then
if lsmod | grep -q nfsv3; then
mount -t nfs $IP:/srv/pxe/storage /mnt/nfs/storage -o rw,nolock 2>/dev/null && { mkdir -p /mnt/nfs/storage/client-$MAC/changes/home; CHANGES="/storage/client-$MAC"; }
else
mount -t nfs4 $IP:/srv/pxe/storage /mnt/nfs/storage -o rw,nolock 2>/dev/null && { mkdir -p /mnt/nfs/storage/client-$MAC/changes/home; CHANGES="/storage/client-$MAC"; }
fi
fi
elif [ $ISO ]; then SGNDEV=/mnt/isoloop
locate -e $FROM && { BOOTDEV=/mnt/$DEV; mkdir /mnt/isoloop; mount -o loop /mnt/$DEV/$LPTH /mnt/isoloop; }
else
if [ $FROM ]; then
locate -e $FROM/$FOLDER/$SGN
if [ $? -eq 0 ]; then
DIR=`echo $LPTH | rev | cut -d/ -f3- | rev`; [ $DIR ] && FOLDER=$DIR/$FOLDER
else
echo $i"from= cheatcode is incorrect, press enter to search through all devices"
read -s; search -e $FOLDER/$SGN
fi
else
search -e $FOLDER/$SGN || lazy -e $FOLDER/$SGN
fi
SGNDEV=/mnt/$DEV
fi
[ -e $SGNDEV/$FOLDER/$SGN ] && PTH=$SGNDEV/$FOLDER || . fatal
MODDIR=$PTH/modules
BASDIR=${PTH%/*}
echo $i"using Porteus data from $PTH"
debug
# Make all drivers available:
mount -o loop $PTH/base/000-kernel.xzm /opt/000-kernel 2>/dev/null
mount -o bind /opt/000-kernel/usr/lib/modules /usr/lib/modules 2>/dev/null
# Mount a tmpfs for write access
mount -t tmpfs tmpfs /memory
mkdir -p /memory/changes
mkdir -p /memory/work
mkdir -p /memory/copy2ram
mkdir -p /memory/images
if [ $? -ne 0 ]; then echo -e """cant setup union (aufs) - read only filesystem?\nWhen you finish debugging press Ctrl+Alt+Del to reboot."""; sh; fi
# Find modules:
find $PTH/base $PTH/modules -name "*.xzm" 2>/dev/null | egrep -v "$NOLOAD" | sort >/tmp/modules
find $PTH/optional -name "*.xzm" 2>/dev/null | egrep "$LOAD" | sort >>/tmp/modules
# Populate aufs with modules:
umount /lib/modules /opt/000-kernel 2>/dev/null
echo $i"Mounting modules"
for x in `cat /tmp/modules`; do
NAME=`basename $x`; mkdir /memory/images/$NAME; mount -o loop $x /memory/images/$NAME 2>/dev/null
if [ $? -eq 0 ]; then
echo " $m $NAME"
else
echo $i"""Cannot read $NAME - corrupted module?"""; rmdir /memory/images/$NAME
fi
done
echo $i"Bind mounting lib/modules"
mount -no bind /merge/lib/modules /lib/modules 2>/dev/null
debug
echo $i"Creating list of modules OARGS"
# Create a list of modules to be overlayed
for a in $(ls /memory/images/); do a="/memory/images/$a:"; ARGS=$ARGS$a; done
ARGS=`echo $ARGS |sed 's/\(.*\)./\1/'`
debug
# Create an extra empty but mounted writable directory in the overlay
mkdir /memory/images/test
ARGS="$ARGS:/memory/images/test"
echo $i"Creating overlay mounts"
mount -t overlay overlay -o lowerdir=$ARGS,upperdir=/memory/changes,workdir=/memory/work /merge
#mount -t overlay overlay -o lowerdir=$ARGS,upperdir=/memory/changes,workdir=/memory/work /merge
fstab
debug
# Create 7 free loop devices for truecrypt, etc...
echo $i"Creating 7 extra loop devices"
x=`losetup | tail -n1 | cut -d: -f1 | sed 's@/dev/loop@@'`; let y=x+7
while [ $x -le $y ]; do [ -b /dev/loop$y ] && break || mknod /dev/loop$y b 7 $y; let y=y-1; done
cp -af /dev/console /merge/dev
fstab
debug
# Create debug file:
[ -e /tmp/devices ] && { echo "# Recognized devices:" >$livedbg; cat /tmp/devices >>$livedbg; }
[ $BOOTDEV ] && SGNDEV=$BOOTDEV
echo -e "\n# Booting device:\n$SGNDEV\n\n# Porteus data found in:\n$PTH\n\n# Changes are stored in:\n$CHANGES\n\n# Non standard /rootcopy dir:\n$ROOTCOPY\n\n# Modules activated during boot time:" >>$livedbg; cat /tmp/modules >>$livedbg
cp -af $livedbg /merge/var/log/porteus-livedbg
echo $i"changing root directory"
if param noauto; then
for x in `grep /mnt/ /etc/fstab | cut -d/ -f3`; do
mkdir -p /merge/mnt/$x
umount -n /mnt/$x 2>/dev/null && rmdir /mnt/$x
sed -i '/mnt\/'$x'/d' /etc/fstab ### TESTING
done
else
grep /mnt/ /etc/fstab >> /merge/etc/mtab
for x in `grep /mnt/ /etc/fstab | cut -d/ -f3`; do mkdir -p /merge/mnt/$x; mount -n --move /mnt/$x /merge/mnt/$x; rmdir /mnt/$x; done
fi
sed -i 's/ ntfs / ntfs-3g /g' /etc/fstab
cp -f /etc/fstab /merge/etc 2>/dev/null
umount -n /lib/modules 2>/dev/null
rm -r /lib/* /usr/*
debug
echo """live system is ready now - starting Porteus"""
mkdir /merge/mnt/live
if [ $? -eq 0 ]; then
pivot_root /merge /merge/mnt/live
exec bin/chroot . /usr/bin/init "$@" <dev/console >dev/console 2>&1
else
echo -e """!!ERROR!!\nSomething went wrong and I cannot continue.\nPress Ctrl+Alt+Del to reboot."""
sh
fi
Jack there were substantial changes in 05-devel from slackware 14.1 to 14.2 (slackware-current). The most important being everything is compiled against a higher version of gcc. The script makes life easier, no more having to search the slackware-current repo when updating. I am not sure about 07 and 003 as I haven't looked at them yet.
Re: Building Blocks
Posted: 24 Jan 2016, 04:10
by Jack
brokenman wrote:Jack there were substantial changes in 05-devel from slackware 14.1 to 14.2 (slackware-current). The most important being everything is compiled against a higher version of gcc. The script makes life easier, no more having to search the slackware-current repo when updating. I am not sure about 07 and 003 as I haven't looked at them yet.
So on 05-devel I just can't upgrade them to 14.2 (slackware-current)? Cause that what wanted to do.
Re: Building Blocks
Posted: 24 Jan 2016, 22:15
by brokenman
Assuming you are working from Porteus-v3.1 (slackware 14.1) the first step would be to create a bootable system with a slackware 14.2 001-core.xzm module. From there you can begin to create your 002-xorg.xzm module. You can pull any package from slackware-current. You shouldn't need to compile anything in order to get a working 001-core module. Once you have a slackware 14.2 system you can boot into (or chroot into) it, then you can update 05-devel using the script and start compiling. So in short, anything that is compiled that you want to use in your slackware 14.2 porteus should be compiled ON slackware 14.2 porteus.
Here is a handy command to get the file ready for the script.
Code: Select all
ls -1 /mnt/live/memory/images/001-core.xzm/var/log/packages > /tmp/001-pkg.txt
for a in $(cat /tmp/01-pkg.txt); do echo ${a%-*-*-*} >> /tmp/001-name.txt; done
Re: Building Blocks
Posted: 25 Jan 2016, 23:33
by Bogomips
brokenman wrote:The problem is not that the modules don't load on the bottom read only layer. It is that you can't 'activate' a module once your system is up. Since porteus is all about modules, this is not acceptable for me.
Next on the agenda (following categorical statement): Standalone aufs module for 4.1.15
Progress so Far
Reached 'Setup aufs:' stage, at which point require to merge brokenman's overlayfs variant.
Relevant diagnostic output:
Code: Select all
Loop and Squashfs Activated
Make all drivers available:
:loop /mnt/sda9/porteus/base/000-kernel.xzm /opt/000-kernel
/opt/000-kernel/
lib
/opt/000-kernel/lib/
cpp dhcpcd firmware modprobe.d modules
udev
:bind /opt/000-kernel/lib/modules /lib/modules
/lib
ld-uClibc-0.9.33.2.so ld-uClibc.so.0 libblkid.so.1
libblkid.so.1.1.0 libc.so.0 libcom_err.so.2
libcom_err.so.2.1 libe2p.so.2 libe2p.so.2.3
libext2fs.so.2 libext2fs.so.2.4 libgcc_s.so
libgcc_s.so.1 libpthread-0.9.33.2.so libpthread.so.0
libuClibc-0.9.33.2.so libuuid.so.1 libuuid.so.1.3.0
modules
/lib/modules/
4.1.15
/lib/modules/4.1.15/
build kernel modules.alias
modules.alias.bin modules.builtin modules.builtin.bin
modules.dep modules.dep.bin modules.devname
modules.order modules.softdep modules.symbols
modules.symbols.bin source
/lib/modules/4.1.15/kernel/
arch crypto drivers fs kernel
lib mm net security sound
/lib/modules/4.1.15/kernel/lib/
asn1_decoder.ko bch.ko cordic.ko
crc-ccitt.ko crc7.ko crc8.ko
lru_cache.ko lz4 mpi
reed_solomon test-hexdump.ko test_bpf.ko
test_firmware.ko test_module.ko test_user_copy.ko
ts_bm.ko ts_fsm.ko ts_kmp.ko
/lib/modules/4.1.15/kernel/fs/
9p adfs affs afs autofs4
befs bfs binfmt_misc.ko btrfs cachefiles
ceph cifs coda cramfs dlm
ecryptfs efivarfs efs exofs ext2
ext3 ext4 f2fs fat freevxfs
fscache fuse gfs2 hfs hfsplus
hpfs isofs jbd jbd2 jffs2
jfs logfs mbcache.ko minix ncpfs
nfsd nilfs2 nls ntfs ocfs2
omfs overlayfs pstore qnx4 qnx6
quota reiserfs squashfs sysv ubifs
udf ufs xfs
Create symlinks for porteus scripts:
-sf /mnt/sda9/porteus/modules /porteus/modules
-sf /mnt/sda9/porteus/optional /porteus/optional
Symlinks Done. Setup Changes:
/porteus/
modules
optional
/porteus/optional/
/porteus/modules/
Changes processing finished: mkdir -p /memory/changes/mnt/live
Setup aufs:
Not sure if more has to be added or something taken out
One thing else I noticed with fs processing was that when there wasn't an ext2 to modprobe, ext4 got modprobed instead, and this then worked for the ext2 partition.
guest@porteus:~$ md5sum initrd_250116.img
98f90e789c3e45932eb19804cd28d774
initrd_250116.img
Re: Building Blocks
Posted: 26 Jan 2016, 02:03
by wread
This is the best topic I have ever read! I will print it; the title of this little book will be "Thanks Jay!"
I spent many hours doing those things manually! I learned them mostly the hard way....
Regards
Re: Building Blocks
Posted: 26 Jan 2016, 04:08
by Jack
I'm still looking for a few more packages. But when I do try and build 001-core should I boot up in text mode or Graphics mode (MATE)? I want and hope it works. I will supply a list of packages I can't fine.
Re: Building Blocks
Posted: 26 Jan 2016, 19:50
by brokenman
I spent many hours doing those things manually! I learned them mostly the hard way....
Then we could use your experience here definitely. You may also like to reply to my request for 003-kde packages list. It would help the slackware cause. Thanks.
Jack most probably ... it won't boot the first time. Be prepared for that. Boot into text mode and issue:
startx 2> /tmp/startx.log
When it fails you can read the log.
Bogomips it looks good to me. When I get a chance i will upload the overlayfs porteus somewhere in the interest of experimentation. It may be a while though. It can't be called Porteus as it is mostly a lightweight slackware live OS in which you can install slackware packages.
Go forth and conquer.
Re: Building Blocks
Posted: 26 Jan 2016, 20:03
by Jack
@ brokenman,
The following packages were not found. But I found most. * - needs to be converted or compiled. They might be out dated.
httpfs2-------------------------httpfs2-0.1.4-x86_64-1dj.txz
ifenslave-----------------------ifenslave-1.1.0.x86.64-pkg-8.txz
iputils---------------------------iputils-20140519.fad11dc.1.x86.64-pkg-1.txz
laptop-mode-tools----------laptop-mode-tools-1.61-x86_64-1_SBo.tgz
p7zip---------------------------p7zip-15.09-x86_64-1alien.tgz
pptp----------------------------pptp-1.8.0-x86_64-1_SBo.tgz
sshfs-fuse---------------------sshfs-fuse-2.5-x86_64-1alien.tgz
tofrodos-----------------*-----tofrodos-1.7.13.tar.gz
unrar---------------------------unrar-4.2.4-x86_64-1alien.tgz
usm----------------------------usm-3.1.9-noarch-1.txz
webfs--------------------------webfs-1.21.x86.64-pkg-11.txz
wgetpaste--------------------wgetpaste-2.25-noarch-2_SBo.tgz
This was missing in the list brokenman has posted. Can I use an unstripped version?
kernel-firmware-stripped
Do any need to be change?
Re: Building Blocks
Posted: 26 Jan 2016, 23:42
by brokenman
These packages are not essential to get a bootable system however the sources you have chosen are fine.
Can I use an unstripped version?
Yes, just the size will be bigger. That's all. The kernel firmware has been moved into 000-kernel and is no longer stripped.
Do any need to be change?
No
Re: Building Blocks
Posted: 27 Jan 2016, 00:28
by Jack
Why I added those files you had them is the list I downloaded. If I new the minimum amount of packages that all I would use but I don't know for sure.