Page 2 of 2

OverlayFS Porteus

Posted: 06 Sep 2020, 05:19
by neko
@babam
One more:
Please try Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso with persistence mode.
And would you tell me the result.

Thanks.
-----------------------
My test result:
Insert baseModule.xzm into Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso

Code: Select all

% ls
Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso
% mloop Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso
% ls /mnt/loop/porteus/modules
atool.xzm  baseModule.xzm
% cp /mnt/loop/porteus/modules/baseModule.xzm .
% uloop
% mloop baseModule.xzm
% ls /mnt/loop/bin
foo
% cat /mnt/loop/bin/foo
This is the file in baseModule/bin/foo.
% uloop
Boot by the grub2 entry following.
menuentry "5.0rc2=====>Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso<======" {
search --no-floppy --fs-uuid --set=root $BIGuuid
set ISO="/porteus/5.0/rc2/Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso"
loopback loop $ISO
linux (loop)/boot/syslinux/vmlinuz from=UUID:${BIGuuid}$ISO copy2ram kmap=jp \
changes=UUID:${uuid}/porteus/5.0/rc2/persist
initrd (loop)/boot/syslinux/initrd.zstd
}

Code: Select all

% cat /bin/foo
This is the file in baseModule/bin/foo.
% ls
activateModule.xzm
% mloop activateModule.xzm
% cat /mnt/loop/bin/foo
This is the file in activateModule/bin/foo.
% uloop
% sudo activate activateModule.xzm
% cat /bin/foo
This is the file in activateModule/bin/foo.

Reboot by the same grub2 entry.

Code: Select all

% cat /bin/foo
This is the file in baseModule/bin/foo.

OverlayFS Porteus

Posted: 06 Sep 2020, 06:10
by babam
neko,
I have tried it and the results have been explained in my previous post.

Why are the results different with you?

OverlayFS Porteus

Posted: 06 Sep 2020, 09:21
by neko
@babam
Regarding the overlapping of the same file name, this implementation is simplified and deficient compared to AUFS.
Your pointing is exactly pointing out this deficiency.


Thanks.
-----------------
My test2 result:
Boot original Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso with the grub2 entry following.
menuentry "5.0rc2=====>Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso<======" {
search --no-floppy --fs-uuid --set=root $BIGuuid
set ISO="/porteus/5.0/rc2/Porteus-XFCE-v5.0rc2-k5.9rc5-x86_64.iso"
loopback loop $ISO
linux (loop)/boot/syslinux/vmlinuz from=UUID:${BIGuuid}$ISO copy2ram kmap=jp \
changes=UUID:${uuid}/porteus/5.0/rc2/persist
initrd (loop)/boot/syslinux/initrd.zstd
}

Code: Select all

% su
# ls /bin/foo
/bin/ls: cannot access '/bin/foo': No such file or directory
# echo "This file is in CHANGES file." > /bin/foo
# cat /bin/foo
This file is in CHANGES file.
# reboot
Reboot by the same grub2 entry.

Code: Select all

% su
# cat /bin/foo
This file is in CHANGES file.
# ls *.xzm
activateModule.xzm  baseModule.xzm
# activate baseModule.xzm
# cat /bin/foo
This is the file in baseModule/bin/foo.
# activate activateModule.xzm
# cat /bin/foo
This is the file in activateModule/bin/foo.
# deactivate baseModule.xzm
# cat /bin/foo
This file is in CHANGES file.
# reboot
Reboot by the same grub2 entry.

Code: Select all

% su
# cat /bin/foo
This file is in CHANGES file.
# activate activateModule.xzm
# cat /bin/foo
This is the file in activateModule/bin/foo.
# reboot
Reboot by the same grub2 entry.

Code: Select all

% su
# cat /bin/foo
cat: /bin/foo: No such file or directory
#
----------------------------------
Why are the results different with you?
---->
Since the original is not restored, it will be deleted if the original is on the persistence (variation inheritance) file.

AT "setClean" function in "finit" file of initrd for OVFS,
"# mv $x $DIR/$PKG" is commented out.

Code: Select all

% diff -r AUFS-initrd/ OV.initrd/
diff -r AUFS-initrd/cleanup OV.initrd/cleanup
12a13,15
> 
>    mkdir -p union
> 
34,45c37,50
< if [ -e /tmp/changes-exit ]; then
<     echo -e "Your session will be saved in 3 seconds.\nPress space/enter to start doing it now or any other key to skip."; x=3
<     while [ $x -gt 0 ]; do read -s -t1 -n1 ans && break || sleep 1; let x=x-1; done
<     if [ "$ans" = "" ]; then
< 	DEST=`cat /tmp/changes-exit`; NAME=`basename $DEST`; MNAME=/memory/images/changes; FOLDERS=`grep '^/' /union/etc/changes-exit.conf | sed s/^.//g`
< 	echo "saving changes to $NAME - do not power off the PC"
< 	cd /memory/changes; rm -rf var/lock/subsys/* var/run/laptop-mode-tools/* `grep '^!' /union/etc/changes-exit.conf | sed s/^..//g | tr "\n" " "`
< 	for x in `find $FOLDERS -name ".wh.*" 2>/dev/null | sed s/.wh.//g | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; test -e $MNAME/"$x" && rm -rf $MNAME/"$x"; done
< 	for x in `find $MNAME -name ".wh.*" 2>/dev/null | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; wh=`echo $x | sed -e s^$MNAME^^g -e s/.wh.//g`; test -e "$wh" && rm "$x"; done
< 	cp -afu --parents $FOLDERS $MNAME 2>/dev/null
<     fi
< elif grep -q ^memory /var/log/livedbg; then
---
> #if [ -e /tmp/changes-exit ]; then
> #    echo -e "Your session will be saved in 3 seconds.\nPress space/enter to start doing it now or any other key to skip."; x=3
> #    while [ $x -gt 0 ]; do read -s -t1 -n1 ans && break || sleep 1; let x=x-1; done
> #    if [ "$ans" = "" ]; then
> #	DEST=`cat /tmp/changes-exit`; NAME=`basename $DEST`; MNAME=/memory/images/changes; FOLDERS=`grep '^/' /union/etc/changes-exit.conf | sed s/^.//g`
> #	echo "saving changes to $NAME - do not power off the PC"
> #	cd /memory/changes; rm -rf var/lock/subsys/* var/run/laptop-mode-tools/* `grep '^!' /union/etc/changes-exit.conf | sed s/^..//g | tr "\n" " "`
> #	for x in `find $FOLDERS -name ".wh.*" 2>/dev/null | sed s/.wh.//g | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; test -e $MNAME/"$x" && rm -rf $MNAME/"$x"; done
> #	for x in `find $MNAME -name ".wh.*" 2>/dev/null | tr ' ' '@'`; do x=`echo $x | tr '@' ' ' `; wh=`echo $x | sed -e s^$MNAME^^g -e s/.wh.//g`; test -e "$wh" && rm "$x"; done
> #	cp -afu --parents $FOLDERS $MNAME 2>/dev/null
> #    fi
> #elif grep -q ^memory /var/log/livedbg; then
> 
> if grep -q ^memory /var/log/livedbg; then




diff -r AUFS-initrd/finit OV.initrd/finit
44c44,46
< aufs / aufs defaults 0 0
---
> #aufs / aufs defaults 0 0
> rootOV / overlay 0 0
> 
157a160,260
> 
> #======================================
> # setClean
> #--------------------------------------
> setClean()
> {
> #=== 変分反映層
> 	mkdir -p /memory/changes/upperdir
> #=== 変分管理データ層
> 	mkdir -p /memory/changes/workdir
> 
> #--<継承エリア初期化>--#
> local PLACE=/memory/changes/upperdir
> 	rm -rf $PLACE/boot 2>/dev/null
> 	rm -rf $PLACE/dev 2>/dev/null
> 	rm -rf $PLACE/mnt 2>/dev/null
> 	rm -rf $PLACE/proc 2>/dev/null
> 	rm -rf $PLACE/run 2>/dev/null
> 	rm -rf $PLACE/sys 2>/dev/null
> 	rm -rf $PLACE/tmp 2>/dev/null
> 	rm -rf $PLACE/var/lock/* 2>/dev/null
> 	rm -rf $PLACE/var/run 2>/dev/null
> 	rm -rf $PLACE/var/tmp 2>/dev/null
> 	rm -rf $PLACE/var/spool/cron/cron.?????? 2>/dev/null
> 
> #	chown -R guest:users $PLACE/home/guest 2>/dev/null
> 
> #--< activate 後 deactivate 無しで終了した残骸を削除する >--#
> #--- delete symbolic link setting "entity & tag" ---#
> local x
> local DIR
> local PKG
> 	for x in `find $PLACE -name '.act.new.*' `
> 	do
> 		DIR=${x%/*}
> 		PKG=${x##*/}
> 		PKG=${PKG#.act.new.}
> #echo "x=$x DIR=$DIR PKG=$PKG" 
> 		if [ -h $DIR/$PKG ]
> 		then
> 			rm $DIR/$PKG
> 		fi
> 		rm $x
> 	done
> 
> #--- recover original entity ---#
> 	for x in `find $PLACE -name '.act.org.*' `
> 	do
> #		DIR=${x%/*}
> #		PKG=${x##*/}
> #		PKG=${PKG#.act.org.}
> #		mv $x $DIR/$PKG
> 		rm $x
> 	done
> }
> #======================================
> # IncludeModules
> #--------------------------------------
> IncludeModules()
> {
> #=== 変分反映層
> mkdir -p /memory/changes/upperdir
> UPPERDIR=/memory/changes/upperdir
> #=== 変分管理データ層
> mkdir -p /memory/changes/workdir
> WORKDIR=/memory/changes/workdir
> 
> #--<継承エリア初期化>--#
> setClean
> 
> #=== 固定不変層
> LOWLIST=""
> for x in `cat /tmp/modules`;
> do
> 	NAME=`basename $x`;
> 	mkdir /memory/images/$NAME;
> 	mount -o loop $x /memory/images/$NAME 2>/dev/null
> 	if [ $? -eq 0 ];
> 	then
> 		echo "  $m  $NAME";
> 		LOWLIST=/memory/images/${NAME}:${LOWLIST}
> 	else
> 		echo $i"""Cannot read $NAME - corrupted module?""";
> 		rmdir /memory/images/$NAME
> 	fi
> done
> if [ -d /memory/images/CHANGES ]
> then
> 	LOWLIST=/memory/images/CHANGES/upperdir:${LOWLIST}
> 	cp -a /memory/images/CHANGES/workdir/* $WORKDIR/.
> fi
> LOWLIST=`echo $LOWLIST | sed 's/:$//'`
> 
> #echo "mount -t overlay -o upperdir=$UPPERDIR,lowerdir=$LOWLIST,workdir=$WORKDIR overlay /union"
> mount -t overlay -o upperdir=$UPPERDIR,lowerdir=$LOWLIST,workdir=$WORKDIR rootOV /union
> if [ $? -ne 0 ]
> then
> 	sh
> fi
> }
> 



diff -r AUFS-initrd/linuxrc OV.initrd/linuxrc
121a122,124
>     
>     CHNEXIT=""
>     
150,153c153,158
< 		rmdir /memory/changes/mnt/* 2>/dev/null
< 		rm -rf /memory/changes/var/lock/* /var/run/laptop-mode-tools/* /var/spool/cron/cron.??????
< 		for x in `find /memory/changes/var/run -name "*pid" 2>/dev/null`; do rm $x; done
< 		if [ $CHNEXIT = EXIT -o "`egrep -o " changes-ro( |\$)" /proc/cmdline`" ]; then
---
> 
> #		rmdir /memory/changes/mnt/* 2>/dev/null
> #		rm -rf /memory/changes/var/lock/* /var/run/laptop-mode-tools/* /var/spool/cron/cron.??????
> #		for x in `find /memory/changes/var/run -name "*pid" 2>/dev/null`; do rm $x; done
> 
> 		if [ "$CHNEXIT" = EXIT -o "`egrep -o " changes-ro( |\$)" /proc/cmdline`" ]; then
156,160c161,174
< 		    for x in `find /memory/changes -name ".wh.*" | grep -v '.wh..wh..opq' | tr ' ' '@' `; do x=`echo $x | tr '@' ' ' `; cp -a --parents "$x" /var; done
< 		    umount /memory/changes; mount -nt tmpfs -o size=$RAMSIZE tmpfs /memory/changes
< 		    # need to fix busybox bug on 'cp -a' as it does not preserve perms on dirs:
< 		    mv /var/memory/changes/* /memory/changes 2>/dev/null; CHANGES=memory
< 		    chown -R guest /memory/changes/home/guest 2>/dev/null
---
> 
> #		    for x in `find /memory/changes -name ".wh.*" | grep -v '.wh..wh..opq' | tr ' ' '@' `; do x=`echo $x | tr '@' ' ' `; cp -a --parents "$x" /var; done
> #		    umount /memory/changes; mount -nt tmpfs -o size=$RAMSIZE tmpfs /memory/changes
> #		    # need to fix busybox bug on 'cp -a' as it does not preserve perms on dirs:
> #		    mv /var/memory/changes/* /memory/changes 2>/dev/null; CHANGES=memory
> #		    chown -R guest /memory/changes/home/guest 2>/dev/null
> 
> 			setClean
> 			mkdir -p /memory/images/CHANGES
> 			mount -n --move /memory/changes /memory/images/CHANGES
> 			umount /memory/changes;
> 			mount -nt tmpfs -o size=$RAMSIZE tmpfs /memory/changes;
> 			CHANGES=memory
> 
161a176,177
> 
> 
177,179c193,195
< echo $i"creating live filesystem and inserting modules"
< mount -t aufs -o nowarn_perm,xino=/memory/xino/.aufs.xino,br:/memory/changes=rw aufs /union
< if [ $? -ne 0 ]; then echo -e """cant setup union (aufs) - read only filesystem?\nWhen you finish debugging press Ctrl+Alt+Del to reboot."""; sh; fi
---
> #echo $i"creating live filesystem and inserting modules"
> #mount -t aufs -o nowarn_perm,xino=/memory/xino/.aufs.xino,br:/memory/changes=rw aufs /union
> #if [ $? -ne 0 ]; then echo -e """cant setup union (aufs) - read only filesystem?\nWhen you finish debugging press Ctrl+Alt+Del to reboot."""; sh; fi
257,266c273,287
< while read line; do
<     NAME=`basename "$line"`
<     mkdir /memory/images/"$NAME"
<     mount -o loop "$line" /memory/images/"$NAME" 2>/dev/null
<     if [ $? -eq 0 ]; then
< 	echo "  $m  $NAME"; mount -no remount,add:1:/memory/images/"$NAME"=rr aufs /union
<     else
< 	echo $i"""Cannot read $NAME - corrupted module?"""; rmdir /memory/images/"$NAME"
<     fi
< done < /tmp/modules
---
> 
> #while read line; do
> #    NAME=`basename "$line"`
> #    mkdir /memory/images/"$NAME"
> #    mount -o loop "$line" /memory/images/"$NAME" 2>/dev/null
> #    if [ $? -eq 0 ]; then
> #	echo "  $m  $NAME"; mount -no remount,add:1:/memory/images/"$NAME"=rr aufs /union
> #    else
> #	echo $i"""Cannot read $NAME - corrupted module?"""; rmdir /memory/images/"$NAME"
> #    fi
> #done < /tmp/modules
> 
> IncludeModules
> 
> 
270,282c291,303
< if [ -e /tmp/changes-exit ]; then
<     mkdir /memory/images/changes
<     if [ -d $CHNEXIT ]; then
< 	mount -o bind $CHNEXIT/changes /memory/images/changes
<     elif [ -b /dev/mapper/crypt ]; then
< 	mount /dev/mapper/crypt /memory/images/changes
<     else
< 	mount -o loop $CHNEXIT /memory/images/changes
<     fi
<     echo "  $m  changes"; mount -no remount,add:1:/memory/images/changes=ro aufs /union
<     echo $CHNEXIT/changes >>/tmp/modules
<     param changes-ro && rm /tmp/changes-exit
< fi
---
> #if [ -e /tmp/changes-exit ]; then
> #    mkdir /memory/images/changes
> #    if [ -d $CHNEXIT ]; then
> #	mount -o bind $CHNEXIT/changes /memory/images/changes
> #    elif [ -b /dev/mapper/crypt ]; then
> #	mount /dev/mapper/crypt /memory/images/changes
> #    else
> #	mount -o loop $CHNEXIT /memory/images/changes
> #    fi
> #    echo "  $m  changes"; mount -no remount,add:1:/memory/images/changes=ro aufs /union
> #    echo $CHNEXIT/changes >>/tmp/modules
> #    param changes-ro && rm /tmp/changes-exit
> #fi
431a453,455
> 
> 	mkdir -p /union/mnt/live
> 
$

OverlayFS Porteus

Posted: 06 Sep 2020, 23:40
by babam
neko,
The following is a serious issue (not related to persistence mode).

a.xzm containing '/opt/bin/a'
b.xzm containing '/opt/bin/b'

Code: Select all

# ls /opt
dir1   dir2
# activate a.xzm
# ls /opt
bin   dir1   dir2
# ls /opt/bin
a
# activate b.xzm
# ls /opt
bin   dir1   dir2
# ls /opt/bin
b

OverlayFS Porteus

Posted: 07 Sep 2020, 08:40
by neko
@babam
Regarding the overlapping of the same file name, this implementation is simplified and deficient compared to AUFS.
Your new pointing (serious issue) too is exactly pointing out this deficiency.

In order to realize the "ACTIVATE/DEACTIVATE" function easily and at high speed,
the method of ignoring the same name entity of the previous "ACTIVATE" was adopted for overlapping file names and directory names issue.

If possible, avoid the "same name" problems by using a some operation (usage).
The worst one can be avoided by incorporating at boot (including the XZM in ISO or setting the XZM in "extramod" directory).


Thanks.
-----------------
My test3 result:

Code: Select all

% su
# ls
sameDIR1.xzm sameDIR2.xzm
# activate sameDIR1.xzm
# ls /opt
bin@  porteus-scripts/
# ls /opt/bin/
foo1
# cat /opt/bin/foo1
This file is in sameDIR1/opt/bin.
# activate sameDIR2.xzm
# ls /opt
bin@  porteus-scripts/
# ls /opt/bin/
foo2
# cat /opt/bin/foo2
This file is in sameDIR2/opt/bin.
# deactivate sameDIR1.xzm
# ls /opt/
porteus-scripts/
# deactivate sameDIR2.xzm
# ls /opt/
porteus-scripts/
# mkdir /opt/bin
# activate sameDIR1.xzm
# ls /opt/
bin/  porteus-scripts/
# ls /opt/bin/
foo1@
# activate sameDIR2.xzm
# ls /opt/
bin/  porteus-scripts/
# ls /opt/bin/
foo1@  foo2@
# cat /opt/bin/*
This file is in sameDIR1/opt/bin.
This file is in sameDIR2/opt/bin.
#

OverlayFS Porteus

Posted: 09 Sep 2020, 06:07
by babam
neko,
However ov.act.sh and ov.deact.sh are great scripts, they do a pretty good job.

Even with ov.act.sh and ov.deact.sh (with a few modifications) it allows me to activate Porteus modules (google-chrome.xzm, opera.xzm and onlyoffice.xzm) on my Ubuntu system.

Thanks.

OverlayFS Porteus

Posted: 09 Sep 2020, 11:19
by neko
@babam
Thank you for your pointing the "name duplication" issue.
OVFS is a temporary present for absence of AUFS.
(That might have a some issue.)

It is much better for Porteus user to use AUFS.
I wait for Mr. J. R. Okajima come back.
Porteus Kernel Builder (Post by peebee #78435)

Thanks.

OverlayFS Porteus

Posted: 15 Oct 2020, 12:39
by Jappie Toutenhoofd
:worship: Thanks for keeping up with the 5.9RC's and the final :juggler:
Must feel like holidays for you now, now the kernel is final.
Any new challenges on your drawing-table that you can share?
If you have some clear space there, I would like to give you a hint/challenge: Alpine Edge Porteus :oops:

OverlayFS Porteus

Posted: 18 Oct 2020, 01:50
by neko
@"Jappie Toutenhoofd"
I downloaded "alpine-standard-3.12.0-x86_64.iso"
But, I could not understand the relation "between OverlayFS impliment" and "Alpine".
Would you tell me what "Alpine Edge Porteus" is ?

Thanks.

OverlayFS Porteus

Posted: 18 Oct 2020, 14:31
by Jappie Toutenhoofd
@neko , there is no relation between OverlayFS and Alpine, really.
It is just that this Porteus-version with kernel5.9 is really smooth on my Asus-C302. My C302 has only 32Gb HDD so I need a small footprint.
  • The most important plus-points of Porteus (and its derivates) are:
    a) quick boot time
    b) small install size
    c) pre-installed ready-to-use desktop-environments.
  • The strong points of Alpine (Edge)
    a) guick boot time
    b) small install size (even without squash or zstd)
but there are no ready-to-use ISO's for Alpine as far as I know.
So if someone would combine those two distros, I would expect (atleast hope) the qualities would add up.