OverlayFS Porteus

Here is a place for your projects which are not officially supported by the Porteus Team. For example: your own kernel patched with extra features; desktops not included in the standard ISO like Gnome; base modules that are different than the standard ISO, etc...
neko
DEV Team
DEV Team
Posts: 1678
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

OverlayFS Porteus

Post#16 by neko » 06 Sep 2020, 05:19

@babam
One more:
Please try Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso with persistence mode.
And would you tell me the result.

Thanks.
-----------------------
My test result:
Insert baseModule.xzm into Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso

Code: Select all

% ls
Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso
% mloop Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso
% ls /mnt/loop/porteus/modules
atool.xzm  baseModule.xzm
% cp /mnt/loop/porteus/modules/baseModule.xzm .
% uloop
% mloop baseModule.xzm
% ls /mnt/loop/bin
foo
% cat /mnt/loop/bin/foo
This is the file in baseModule/bin/foo.
% uloop
Boot by the grub2 entry following.
menuentry "5.0rc2=====>Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso<======" {
search --no-floppy --fs-uuid --set=root $BIGuuid
set ISO="/porteus/5.0/rc2/Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso"
loopback loop $ISO
linux (loop)/boot/syslinux/vmlinuz from=UUID:${BIGuuid}$ISO copy2ram kmap=jp \
changes=UUID:${uuid}/porteus/5.0/rc2/persist
initrd (loop)/boot/syslinux/initrd.zstd
}

Code: Select all

% cat /bin/foo
This is the file in baseModule/bin/foo.
% ls
activateModule.xzm
% mloop activateModule.xzm
% cat /mnt/loop/bin/foo
This is the file in activateModule/bin/foo.
% uloop
% sudo activate activateModule.xzm
% cat /bin/foo
This is the file in activateModule/bin/foo.

Reboot by the same grub2 entry.

Code: Select all

% cat /bin/foo
This is the file in baseModule/bin/foo.

User avatar
babam
Samurai
Samurai
Posts: 139
Joined: 16 Nov 2016, 10:30
Distribution: Porteus Xfce 3.2.2
Location: Rainy city

OverlayFS Porteus

Post#17 by babam » 06 Sep 2020, 06:10

neko,
I have tried it and the results have been explained in my previous post.

Why are the results different with you?
Sorry, my English is bad.

neko
DEV Team
DEV Team
Posts: 1678
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

OverlayFS Porteus

Post#18 by neko » 06 Sep 2020, 09:21

@babam
Regarding the overlapping of the same file name, this implementation is simplified and deficient compared to AUFS.
Your pointing is exactly pointing out this deficiency.


Thanks.
-----------------
My test2 result:
Boot original Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso with the grub2 entry following.
menuentry "5.0rc2=====>Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso<======" {
search --no-floppy --fs-uuid --set=root $BIGuuid
set ISO="/porteus/5.0/rc2/Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso"
loopback loop $ISO
linux (loop)/boot/syslinux/vmlinuz from=UUID:${BIGuuid}$ISO copy2ram kmap=jp \
changes=UUID:${uuid}/porteus/5.0/rc2/persist
initrd (loop)/boot/syslinux/initrd.zstd
}

Code: Select all

% su
# ls /bin/foo
/bin/ls: cannot access '/bin/foo': No such file or directory
# echo "This file is in CHANGES file." > /bin/foo
# cat /bin/foo
This file is in CHANGES file.
# reboot
Reboot by the same grub2 entry.

Code: Select all

% su
# cat /bin/foo
This file is in CHANGES file.
# ls *.xzm
activateModule.xzm  baseModule.xzm
# activate baseModule.xzm
# cat /bin/foo
This is the file in baseModule/bin/foo.
# activate activateModule.xzm
# cat /bin/foo
This is the file in activateModule/bin/foo.
# deactivate baseModule.xzm
# cat /bin/foo
This file is in CHANGES file.
# reboot
Reboot by the same grub2 entry.

Code: Select all

% su
# cat /bin/foo
This file is in CHANGES file.
# activate activateModule.xzm
# cat /bin/foo
This is the file in activateModule/bin/foo.
# reboot
Reboot by the same grub2 entry.

Code: Select all

% su
# cat /bin/foo
cat: /bin/foo: No such file or directory
#
----------------------------------
Why are the results different with you?
---->
Since the original is not restored, it will be deleted if the original is on the persistence (variation inheritance) file.

AT "setClean" function in "finit" file of initrd for OVFS,
"# mv $x $DIR/$PKG" is commented out.

Code: Select all

% diff -r AUFS-initrd/ OV.initrd/
diff -r AUFS-initrd/cleanup OV.initrd/cleanup
12a13,15
> 
>    mkdir -p union
> 
34,45c37,50
< if [ -e /tmp/changes-exit ]; then
<     echo -e "Your session will be saved in 3 seconds.\nPress space/enter to start doing it now or any other key to skip."; x=3
<     while [ $x -gt 0 ]; do read -s -t1 -n1 ans && break || sleep 1; let x=x-1; done
<     if [ "$ans" = "" ]; then
< 	DEST=`cat /tmp/changes-exit`; NAME=`basename $DEST`; MNAME=/memory/images/changes; FOLDERS=`grep '^/' /union/etc/changes-exit.conf | sed s/^.//g`
< 	echo "saving changes to $NAME - do not power off the PC"
< 	cd /memory/changes; rm -rf var/lock/subsys/* var/run/laptop-mode-tools/* `grep '^!' /union/etc/changes-exit.conf | sed s/^..//g | tr "\n" " "`
< 	for x in `find $FOLDERS -name ".wh.*" 2>/dev/null | sed s/.wh.//g | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; test -e $MNAME/"$x" && rm -rf $MNAME/"$x"; done
< 	for x in `find $MNAME -name ".wh.*" 2>/dev/null | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; wh=`echo $x | sed -e s^$MNAME^^g -e s/.wh.//g`; test -e "$wh" && rm "$x"; done
< 	cp -afu --parents $FOLDERS $MNAME 2>/dev/null
<     fi
< elif grep -q ^memory /var/log/livedbg; then
---
> #if [ -e /tmp/changes-exit ]; then
> #    echo -e "Your session will be saved in 3 seconds.\nPress space/enter to start doing it now or any other key to skip."; x=3
> #    while [ $x -gt 0 ]; do read -s -t1 -n1 ans && break || sleep 1; let x=x-1; done
> #    if [ "$ans" = "" ]; then
> #	DEST=`cat /tmp/changes-exit`; NAME=`basename $DEST`; MNAME=/memory/images/changes; FOLDERS=`grep '^/' /union/etc/changes-exit.conf | sed s/^.//g`
> #	echo "saving changes to $NAME - do not power off the PC"
> #	cd /memory/changes; rm -rf var/lock/subsys/* var/run/laptop-mode-tools/* `grep '^!' /union/etc/changes-exit.conf | sed s/^..//g | tr "\n" " "`
> #	for x in `find $FOLDERS -name ".wh.*" 2>/dev/null | sed s/.wh.//g | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; test -e $MNAME/"$x" && rm -rf $MNAME/"$x"; done
> #	for x in `find $MNAME -name ".wh.*" 2>/dev/null | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; wh=`echo $x | sed -e s^$MNAME^^g -e s/.wh.//g`; test -e "$wh" && rm "$x"; done
> #	cp -afu --parents $FOLDERS $MNAME 2>/dev/null
> #    fi
> #elif grep -q ^memory /var/log/livedbg; then
> 
> if grep -q ^memory /var/log/livedbg; then




diff -r AUFS-initrd/finit OV.initrd/finit
44c44,46
< aufs / aufs defaults 0 0
---
> #aufs / aufs defaults 0 0
> rootOV / overlay 0 0
> 
157a160,260
> 
> #======================================
> # setClean
> #--------------------------------------
> setClean()
> {
> #=== 変分反映層
> 	mkdir -p /memory/changes/upperdir
> #=== 変分管理データ層
> 	mkdir -p /memory/changes/workdir
> 
> #--<継承エリア初期化>--#
> local PLACE=/memory/changes/upperdir
> 	rm -rf $PLACE/boot 2>/dev/null
> 	rm -rf $PLACE/dev 2>/dev/null
> 	rm -rf $PLACE/mnt 2>/dev/null
> 	rm -rf $PLACE/proc 2>/dev/null
> 	rm -rf $PLACE/run 2>/dev/null
> 	rm -rf $PLACE/sys 2>/dev/null
> 	rm -rf $PLACE/tmp 2>/dev/null
> 	rm -rf $PLACE/var/lock/* 2>/dev/null
> 	rm -rf $PLACE/var/run 2>/dev/null
> 	rm -rf $PLACE/var/tmp 2>/dev/null
> 	rm -rf $PLACE/var/spool/cron/cron.?????? 2>/dev/null
> 
> #	chown -R guest:users $PLACE/home/guest 2>/dev/null
> 
> #--< activate 後 deactivate 無しで終了した残骸を削除する >--#
> #--- delete symbolic link setting "entity & tag" ---#
> local x
> local DIR
> local PKG
> 	for x in `find $PLACE -name '.act.new.*' `
> 	do
> 		DIR=${x%/*}
> 		PKG=${x##*/}
> 		PKG=${PKG#.act.new.}
> #echo "x=$x DIR=$DIR PKG=$PKG" 
> 		if [ -h $DIR/$PKG ]
> 		then
> 			rm $DIR/$PKG
> 		fi
> 		rm $x
> 	done
> 
> #--- recover original entity ---#
> 	for x in `find $PLACE -name '.act.org.*' `
> 	do
> #		DIR=${x%/*}
> #		PKG=${x##*/}
> #		PKG=${PKG#.act.org.}
> #		mv $x $DIR/$PKG
> 		rm $x
> 	done
> }
> #======================================
> # IncludeModules
> #--------------------------------------
> IncludeModules()
> {
> #=== 変分反映層
> mkdir -p /memory/changes/upperdir
> UPPERDIR=/memory/changes/upperdir
> #=== 変分管理データ層
> mkdir -p /memory/changes/workdir
> WORKDIR=/memory/changes/workdir
> 
> #--<継承エリア初期化>--#
> setClean
> 
> #=== 固定不変層
> LOWLIST=""
> for x in `cat /tmp/modules`;
> do
> 	NAME=`basename $x`;
> 	mkdir /memory/images/$NAME;
> 	mount -o loop $x /memory/images/$NAME 2>/dev/null
> 	if [ $? -eq 0 ];
> 	then
> 		echo "  $m  $NAME";
> 		LOWLIST=/memory/images/${NAME}:${LOWLIST}
> 	else
> 		echo $i"""Cannot read $NAME - corrupted module?""";
> 		rmdir /memory/images/$NAME
> 	fi
> done
> if [ -d /memory/images/CHANGES ]
> then
> 	LOWLIST=/memory/images/CHANGES/upperdir:${LOWLIST}
> 	cp -a /memory/images/CHANGES/workdir/* $WORKDIR/.
> fi
> LOWLIST=`echo $LOWLIST | sed 's/:$//'`
> 
> #echo "mount -t overlay -o upperdir=$UPPERDIR,lowerdir=$LOWLIST,workdir=$WORKDIR overlay /union"
> mount -t overlay -o upperdir=$UPPERDIR,lowerdir=$LOWLIST,workdir=$WORKDIR rootOV /union
> if [ $? -ne 0 ]
> then
> 	sh
> fi
> }
> 



diff -r AUFS-initrd/linuxrc OV.initrd/linuxrc
121a122,124
>     
>     CHNEXIT=""
>     
150,153c153,158
< 		rmdir /memory/changes/mnt/* 2>/dev/null
< 		rm -rf /memory/changes/var/lock/* /var/run/laptop-mode-tools/* /var/spool/cron/cron.??????
< 		for x in `find /memory/changes/var/run -name "*pid" 2>/dev/null`; do rm $x; done
< 		if [ $CHNEXIT = EXIT -o "`egrep -o " changes-ro( |\$)" /proc/cmdline`" ]; then
---
> 
> #		rmdir /memory/changes/mnt/* 2>/dev/null
> #		rm -rf /memory/changes/var/lock/* /var/run/laptop-mode-tools/* /var/spool/cron/cron.??????
> #		for x in `find /memory/changes/var/run -name "*pid" 2>/dev/null`; do rm $x; done
> 
> 		if [ "$CHNEXIT" = EXIT -o "`egrep -o " changes-ro( |\$)" /proc/cmdline`" ]; then
156,160c161,174
< 		    for x in `find /memory/changes -name ".wh.*" | grep -v '.wh..wh..opq' | tr ' ' '@' `; do x=`echo $x | tr '@' ' ' `; cp -a --parents "$x" /var; done
< 		    umount /memory/changes; mount -nt tmpfs -o size=$RAMSIZE tmpfs /memory/changes
< 		    # need to fix busybox bug on 'cp -a' as it does not preserve perms on dirs:
< 		    mv /var/memory/changes/* /memory/changes 2>/dev/null; CHANGES=memory
< 		    chown -R guest /memory/changes/home/guest 2>/dev/null
---
> 
> #		    for x in `find /memory/changes -name ".wh.*" | grep -v '.wh..wh..opq' | tr ' ' '@' `; do x=`echo $x | tr '@' ' ' `; cp -a --parents "$x" /var; done
> #		    umount /memory/changes; mount -nt tmpfs -o size=$RAMSIZE tmpfs /memory/changes
> #		    # need to fix busybox bug on 'cp -a' as it does not preserve perms on dirs:
> #		    mv /var/memory/changes/* /memory/changes 2>/dev/null; CHANGES=memory
> #		    chown -R guest /memory/changes/home/guest 2>/dev/null
> 
> 			setClean
> 			mkdir -p /memory/images/CHANGES
> 			mount -n --move /memory/changes /memory/images/CHANGES
> 			umount /memory/changes;
> 			mount -nt tmpfs -o size=$RAMSIZE tmpfs /memory/changes;
> 			CHANGES=memory
> 
161a176,177
> 
> 
177,179c193,195
< echo $i"creating live filesystem and inserting modules"
< mount -t aufs -o nowarn_perm,xino=/memory/xino/.aufs.xino,br:/memory/changes=rw aufs /union
< if [ $? -ne 0 ]; then echo -e """cant setup union (aufs) - read only filesystem?\nWhen you finish debugging press Ctrl+Alt+Del to reboot."""; sh; fi
---
> #echo $i"creating live filesystem and inserting modules"
> #mount -t aufs -o nowarn_perm,xino=/memory/xino/.aufs.xino,br:/memory/changes=rw aufs /union
> #if [ $? -ne 0 ]; then echo -e """cant setup union (aufs) - read only filesystem?\nWhen you finish debugging press Ctrl+Alt+Del to reboot."""; sh; fi
257,266c273,287
< while read line; do
<     NAME=`basename "$line"`
<     mkdir /memory/images/"$NAME"
<     mount -o loop "$line" /memory/images/"$NAME" 2>/dev/null
<     if [ $? -eq 0 ]; then
< 	echo "  $m  $NAME"; mount -no remount,add:1:/memory/images/"$NAME"=rr aufs /union
<     else
< 	echo $i"""Cannot read $NAME - corrupted module?"""; rmdir /memory/images/"$NAME"
<     fi
< done < /tmp/modules
---
> 
> #while read line; do
> #    NAME=`basename "$line"`
> #    mkdir /memory/images/"$NAME"
> #    mount -o loop "$line" /memory/images/"$NAME" 2>/dev/null
> #    if [ $? -eq 0 ]; then
> #	echo "  $m  $NAME"; mount -no remount,add:1:/memory/images/"$NAME"=rr aufs /union
> #    else
> #	echo $i"""Cannot read $NAME - corrupted module?"""; rmdir /memory/images/"$NAME"
> #    fi
> #done < /tmp/modules
> 
> IncludeModules
> 
> 
270,282c291,303
< if [ -e /tmp/changes-exit ]; then
<     mkdir /memory/images/changes
<     if [ -d $CHNEXIT ]; then
< 	mount -o bind $CHNEXIT/changes /memory/images/changes
<     elif [ -b /dev/mapper/crypt ]; then
< 	mount /dev/mapper/crypt /memory/images/changes
<     else
< 	mount -o loop $CHNEXIT /memory/images/changes
<     fi
<     echo "  $m  changes"; mount -no remount,add:1:/memory/images/changes=ro aufs /union
<     echo $CHNEXIT/changes >>/tmp/modules
<     param changes-ro && rm /tmp/changes-exit
< fi
---
> #if [ -e /tmp/changes-exit ]; then
> #    mkdir /memory/images/changes
> #    if [ -d $CHNEXIT ]; then
> #	mount -o bind $CHNEXIT/changes /memory/images/changes
> #    elif [ -b /dev/mapper/crypt ]; then
> #	mount /dev/mapper/crypt /memory/images/changes
> #    else
> #	mount -o loop $CHNEXIT /memory/images/changes
> #    fi
> #    echo "  $m  changes"; mount -no remount,add:1:/memory/images/changes=ro aufs /union
> #    echo $CHNEXIT/changes >>/tmp/modules
> #    param changes-ro && rm /tmp/changes-exit
> #fi
431a453,455
> 
> 	mkdir -p /union/mnt/live
> 
$

User avatar
babam
Samurai
Samurai
Posts: 139
Joined: 16 Nov 2016, 10:30
Distribution: Porteus Xfce 3.2.2
Location: Rainy city

OverlayFS Porteus

Post#19 by babam » 06 Sep 2020, 23:40

neko,
The following is a serious issue (not related to persistence mode).

a.xzm containing '/opt/bin/a'
b.xzm containing '/opt/bin/b'

Code: Select all

# ls /opt
dir1   dir2
# activate a.xzm
# ls /opt
bin   dir1   dir2
# ls /opt/bin
a
# activate b.xzm
# ls /opt
bin   dir1   dir2
# ls /opt/bin
b
Sorry, my English is bad.

neko
DEV Team
DEV Team
Posts: 1678
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

OverlayFS Porteus

Post#20 by neko » 07 Sep 2020, 08:40

@babam
Regarding the overlapping of the same file name, this implementation is simplified and deficient compared to AUFS.
Your new pointing (serious issue) too is exactly pointing out this deficiency.

In order to realize the "ACTIVATE/DEACTIVATE" function easily and at high speed,
the method of ignoring the same name entity of the previous "ACTIVATE" was adopted for overlapping file names and directory names issue.

If possible, avoid the "same name" problems by using a some operation (usage).
The worst one can be avoided by incorporating at boot (including the XZM in ISO or setting the XZM in "extramod" directory).


Thanks.
-----------------
My test3 result:

Code: Select all

% su
# ls
sameDIR1.xzm sameDIR2.xzm
# activate sameDIR1.xzm
# ls /opt
bin@  porteus-scripts/
# ls /opt/bin/
foo1
# cat /opt/bin/foo1
This file is in sameDIR1/opt/bin.
# activate sameDIR2.xzm
# ls /opt
bin@  porteus-scripts/
# ls /opt/bin/
foo2
# cat /opt/bin/foo2
This file is in sameDIR2/opt/bin.
# deactivate sameDIR1.xzm
# ls /opt/
porteus-scripts/
# deactivate sameDIR2.xzm
# ls /opt/
porteus-scripts/
# mkdir /opt/bin
# activate sameDIR1.xzm
# ls /opt/
bin/  porteus-scripts/
# ls /opt/bin/
foo1@
# activate sameDIR2.xzm
# ls /opt/
bin/  porteus-scripts/
# ls /opt/bin/
foo1@  foo2@
# cat /opt/bin/*
This file is in sameDIR1/opt/bin.
This file is in sameDIR2/opt/bin.
#

User avatar
babam
Samurai
Samurai
Posts: 139
Joined: 16 Nov 2016, 10:30
Distribution: Porteus Xfce 3.2.2
Location: Rainy city

OverlayFS Porteus

Post#21 by babam » 09 Sep 2020, 06:07

neko,
However ov.act.sh and ov.deact.sh are great scripts, they do a pretty good job.

Even with ov.act.sh and ov.deact.sh (with a few modifications) it allows me to activate Porteus modules (google-chrome.xzm, opera.xzm and onlyoffice.xzm) on my Ubuntu system.

Thanks.
Sorry, my English is bad.

neko
DEV Team
DEV Team
Posts: 1678
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

OverlayFS Porteus

Post#22 by neko » 09 Sep 2020, 11:19

@babam
Thank you for your pointing the "name duplication" issue.
OVFS is a temporary present for absence of AUFS.
(That might have a some issue.)

It is much better for Porteus user to use AUFS.
I wait for Mr. J. R. Okajima come back.
Porteus Kernel Builder (Post by peebee #78435)

Thanks.

Jappie Toutenhoofd
Ronin
Ronin
Posts: 3
Joined: 27 Mar 2020, 11:55
Distribution: overlayFS k5.9

OverlayFS Porteus

Post#23 by Jappie Toutenhoofd » 15 Oct 2020, 12:39

:worship: Thanks for keeping up with the 5.9RC's and the final :juggler:
Must feel like holidays for you now, now the kernel is final.
Any new challenges on your drawing-table that you can share?
If you have some clear space there, I would like to give you a hint/challenge: Alpine Edge Porteus :oops:

neko
DEV Team
DEV Team
Posts: 1678
Joined: 09 Feb 2013, 09:55
Distribution: APorteus-FVWM-ja-x86_64.iso
Location: japan

OverlayFS Porteus

Post#24 by neko » 18 Oct 2020, 01:50

@"Jappie Toutenhoofd"
I downloaded "alpine-standard-3.12.0-x86_64.iso"
But, I could not understand the relation "between OverlayFS impliment" and "Alpine".
Would you tell me what "Alpine Edge Porteus" is ?

Thanks.

Jappie Toutenhoofd
Ronin
Ronin
Posts: 3
Joined: 27 Mar 2020, 11:55
Distribution: overlayFS k5.9

OverlayFS Porteus

Post#25 by Jappie Toutenhoofd » 18 Oct 2020, 14:31

@neko , there is no relation between OverlayFS and Alpine, really.
It is just that this Porteus-version with kernel5.9 is really smooth on my Asus-C302. My C302 has only 32Gb HDD so I need a small footprint.
  • The most important plus-points of Porteus (and its derivates) are:
    a) quick boot time
    b) small install size
    c) pre-installed ready-to-use desktop-environments.
  • The strong points of Alpine (Edge)
    a) guick boot time
    b) small install size (even without squash or zstd)
but there are no ready-to-use ISO's for Alpine as far as I know.
So if someone would combine those two distros, I would expect (atleast hope) the qualities would add up.

Post Reply