container file system advice

[rych]

How do I go about creating my own container image file, similar in spirit to porteus save.dat I guess, that I can mount and unmount and work with like a file system. I'd appreciate a guidance into the search results I'm getting: Docker, overlay fs, overlay2, aufs, etc. or simply doing what's described here: https://unix.stackexchange.com/question ... -container

The image file could be quite large and will be living permanently on a host Windows NTFS disk. As I boot into porteus, I'd mount it and use a normal linux file system to save arbitrary data, not the porteus "changes".

[Ed_P]

Why? I save files I download and backup directly to my NTFS file systems.

[rych]

same reasons that justify save.dat I guess: symbolic links, linux permissions -- all features of an ext4 file system encapsulated in an NTFS file. Plus, I'm going to leave it on the host between sessions: content hidden from Windows search, perhaps even encrypted.

[Ed_P]

Oh. Ok.

[wread]

@rych
I do not have a changes file. I make a module of my changes every time I build or install a new application.

Then I put them all together in a folder named Settings, contained in modules. Then I have:
myDolphin.xzm
myDTSettings.xzm (desktop settings)
myKBSettings.xzm (keyboard settings)
myKDEConnect.xzm (cellphone control)
myKingSoftSettings.xzm (office programs settings)
myKPatSettings.xzm (solitarie game settings)
myLocalTime.xzm (local time and zone settings)
myLoginSound.xzm (login sound setting)
myMCSettings.xm(midnight commander settings)
myPDFEditorSettings.xzm
myRcLocalSettings.xzm
myRVSettings.xzm (network access settings)
myWiFiSettings.xzm

It is not straight forward to build such modules. You must hack a little bit, but they load faster than the changes or the rootcopy folder. And ... I use porteus as root....

Geschmack Sache, sagte der Affe! (It is a matter of taste, said the monkey!)

Cheers!

[brokenman]

You can create an image file like so:

Code: Select all

dd if=/dev/zero of=/tmp/myimage.dat bs=1024 count=10000
mkfs.ext4 /tmp/myimage.dat

This will create a 10MB image.

[tome]

Some advices to create quickly large file, maybe on NTFS also: https://stackoverflow.com/questions/257 ... nux-system

[rych]

Over the years I've come to the conclusion that the best container file in Porteus is xzm module

wread, I've seen you mentioning your method in posts, could you share your scripts please or help me write my own?

I do not have a changes file. I make a module of my changes every time I build or install a new application.

...

It is not straight forward to build such modules. You must hack a little bit, but they load faster than the changes or the rootcopy folder. And ... I use porteus as root....

For example, I want to apply your method to my Firefox Profile. Let's say my Firefox has been set up to load a profile located at /tmp/FirefoxProfile. The idea is that that folder doesn't exist until I activate my module FirefoxProfile.xzm. Running Firefox is going to read and write (changes) to that folder. On quitting Firefox a script is to be run that creates a new FirefoxProfile.xzm from the modified folder /tmp/FirefoxProfile, deactivates and deletes the old FirefoxProfile.xzm, deletes the folder /tmp/FirefoxProfile, preserves the new FirefoxProfile.xzm.

In particular, how do you make an xzm module from /tmp/FirefoxProfile preserving the directory tree like that, so that it expands onto the same path when activated? If I use

Code: Select all

mksquashfs -keep-as-directory

it still doesn't store the /tmp/ part? Or do you use something like

Code: Select all

mksquashfs -no-strip
mksquashfs -tarstyle

( https://manpages.debian.org/unstable/sq ... .1.en.html )

Also, what would be a difference, if any, between /tmp/FirefoxProfile and /mnt/live/tmp/FirefoxProfile?

[Ed_P]

This is how I make a module rych.

Code: Select all

#!/bin/sh
if [ `whoami` != "root" ]; then
   echo -e "Enter root's password\033[1;31m"
   su -c "sh $0 $1"
   exit
fi
echo -e "\033[0m"; echo -en "\033]0;Make Mod\a"

if [ -d /tmp/mod/ ]; then
   rm -rf /tmp/mod
fi
mkdir -p /tmp/mod
cp -a --parents /lib/firmware/iwlwifi-QuZ-a0-hr-b0-71.ucode  /tmp/mod/
cp -a --parents /lib/firmware/iwlwifi-QuZ-a0-jf-b0-71.ucode  /tmp/mod/
dir2xzm   /tmp/mod/                   /tmp/DellInspiron14wifi.xzm
rm    -rf /tmp/mod/ && echo && ls -sh /tmp/*.xzm
read

Added in 10 hours 28 minutes 54 seconds:
Another option I use, create a save.dat file, make all your changes to the system, exit Porteus (I use the cheatcode changes=EXIT for this), rename the save.dat file to be save.xzm and put it in your modules folder.

[rych]

This is how I make a module rych.

-- can't we use symbolic links, we actually have to copy stuff to the mod folder?

Another option I use, create a save.dat file, make all your changes to the system, exit Porteus (I use the cheatcode changes=EXIT for this), rename the save.dat file to be save.xzm and put it in your modules folder.

-- when do you get the opportunity to turn save.dat into a module and move it when the system already shut down? Your changes are written at the very end. So I guess, you have an early startup script that quickly finds save.dat, renames and moves it to modules folder before porteus starts loading modules?

[wread]

@rych
I will explain how I store my changes "per application" in Porteus.

Normally, settings are stored in a hidden file, a file starting with a point that does not show in the home screen of the user. The trick is to find your selected settings in one of those pointed files and make a xzm module with it.

The next time you start as always fresh, mySettings module will be also installed, voila!

That was the theory, now a practice example:
A couple of weeks ago, I installed the application "Ookla Speed Test". After entering the required settings and before login out I went to "root>(switched to view hidden files)>.config>(and bingo!)ookla>speedtest-cli.json". In this last file were my settings stored.

Now let's make the xzm module to it: Folder>New(rename it "myOoklaSettings")>Folder>New(rename it "root")>Folder>New>(rename it ".config")>Folder>New(rename it "ookla", then open that folder and copy speedtest-cli.json into it. Now convert dir2xzm on myOoklaSettings and you are done!

Fine bussinnes!!

[rych]

wread, Ed_P, you and the official /opt/porteus-scripts/update-module copy stuff into a temporary folder before turning it into a module. However, the current version of mksquashfs has the -no-strip parameter which preserves the full path. So we can run it on the actual mounted union directory we're trying to preserve in the module being updated/rewritten. For example in my case, after closing Firefox I simply run:

Code: Select all

COMP_ZSTD="-comp zstd -b 256K -Xcompression-level 22"
mksquashfs /root/.mozilla/firefox/FirefoxProfile $PORTDIR/optional/FirefoxProfile.xzm $COMP_ZSTD -noappend -no-strip

then remove the /root/.mozilla/firefox/FirefoxProfile (or not save it on EXIT:)

Next time I activate the optional/FirefoxProfile.xzm and it expands to /root/.mozilla/firefox/FirefoxProfile

[Ed_P]

the current version of mksquashfs has the -no-strip parameter which preserves the full path. So we can run it on the actual mounted union directory we're trying to preserve in the module

And do the current Porteus 5.0 ISOs have this version of mksquashfs?

[rych]

do the current Porteus 5.0 ISOs have this version of mksquashfs

Apparently so, as I'm using it and didn't have to install anything extra. Reportedly it was introduced upstream in squashfs-tools a year ago: https://lore.kernel.org/lkml/CAB3woddJs ... gmail.com/ -- search on page for the word "strip"

[rych]

I've noticed that dropping -Xcompression-level 22 from

Code: Select all

COMP_ZSTD="-comp zstd -b 256K -Xcompression-level 22"

, thereby using the unknown default value results in a faster compression and only 0.5% increase in size. Actually even using -Xcompression-level 1 the compression is very fast and only 10% larger file. Does anyone have any info on a recommended compression-level for zstd? Otherwise, will be using the unknown default value