[SOLVED] An alternative "changes" cheat code

Here is a place for your projects which are not officially supported by the Porteus Team. For example: your own kernel patched with extra features; desktops not included in the standard ISO like Gnome; base modules that are different than the standard ISO, etc...
User avatar
Thor
Contributor
Contributor
Posts: 18
Joined: 03 Oct 2012, 01:12
Location: Norway

[SOLVED] An alternative "changes" cheat code

Post#1 by Thor » 19 Oct 2012, 22:14

An alternative "changes" cheat code

Most useful for people on slow disks or low income so that they keep their hard earned usb and ssds alive as long as possible :p.. (Is a 486_64 module)

boot and add cheat code "saveonexit"

Changes will only be saved once each session (can also be executed from desktop, but that is done manually). Automatically during shutdown changes In home-folders for guest and root with file size Less then approx 100kb are saved into a module and placed in the module folders where you loaded porteus from. Typically files of use under 100kb would be stuff like desktop-links, bookmarks, small scripts, desktop apaerance, language settings and so on..
Can also be useful for setup of kiosk and to keep systems cleaner. Sort of a preset of "magicfolder" without the binding to a physical disk that magic includes and that can slow system down via slow disks and many changes...and without the "totality (of stuff I'd rather kept unchanged)" that "changes" brings

https://docs.google.com/file/d/0B9BJM4g ... edit?pli=1

User avatar
fanthom
Moderator Team
Moderator Team
Posts: 5666
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland
Contact:

Re: An alternative "changes" cheat code

Post#2 by fanthom » 21 Oct 2012, 10:24

@Thor

i think i like it (must do some tests first) but would like to avoid creating another cheat as we have many already.
what about specifying xzm module as a destination in standard 'changes=' cheat?
example:
'changes=/mnt/sdb2/porteus/modules/changes.xzm'
if linuxrc find that destination is a module then everything is saved in the memory as in 'always fresh' mode.
(/porteus/modules is not obligatory, xzm could be saved anywhere but then users would have to load it with 'extramod=' or 'load=' cheats during next boot)
the path to the module could be exported to /mnt/live/tmp/changes file (i like /mnt/live as it's outside of aufs so any user damage made to it does not persist the reboot).

if rc.6 (during halt/shutdown) find /mnt/live/tmp/changes then:
a) unpacks the module
b) find and delete all files which are marked as .wh. in /mnt/live/memory/changes
c) copy all files (except .wh.) from /mnt/live/changes/{bin,etc,home,lib,lib64,opt,root,sbin,usr,var} to the unpacked module
d) create new module and replace the old one

as you know the only problem are big files and squashing them in the module would take a long time.
there is no direct solution for that (i would like to avoid hardcoding any size limit) but all big downloads in 99% of cases lands in $HOME.
we could advise users to use 'changes=something.xzm" plus optionally "magic folders" set on their home folder to solve this problem (magic folders create a layer above aufs so anything saved on them does not go to aufs rw branch)

pros:
a) Porteus with 'changes=' cheat is faster than ever before as everything is stored in the memory as in 'always fres' mode
b) nothing is is written on a device until shutdown (less I/O banging your flash drive)

cons:
a) user may lose whole session in case of power failure or porteus hang (both rare cases).

so far Porteus was able to save changes on a device (/dev/sdXY), in a folder (/mnt/sdXY/some_folder), in a file (/mnt/sdXY/save.dat) and now it also will be possible to do it inside the module :)

thanks for this nice idea Thor.

EDIT:\\

i may have a solution on big files and long squashing/unsquashing problem:
'changes=' cheat would work as previously (no saving to xzm) but destination save.dat/folder/device would be inserted to aufs as read-only (same as module: =rr) while read-write (=rw) branch will be set on tmpfs.
this way rc.6 (or rather cleanup script from initrd after dismantling of /union) will have to do simple 'cp -a' operation as in reality destination file/folder/device is read-write :)
this way we wont have to waste resources on unsquash/squash: just dump all changes from RAM to a device at the very last stage of rebooting.
that should be fast enough even for large files.

need to do some testing if that works at all (i'm 99% sure it does), but new syntax would look like:
'changes=EXIT:/mnt/sdXY/some-folder'
so linuxrc will know that:
a) /mnt/sdXY/some-folder needs to be treated as a module (read only for aufs)
b) insert it as very last in the queue
c) set =rw branch in tmpfs (like in always fresh mode)

ofc without EXIT: string changes would be saved as normal (in the real time).

discussion is still opened so please share other ideas how to handle "changes on exit" task the best way we can.
Please add [Solved] to your thread title if the solution was found.

User avatar
Thor
Contributor
Contributor
Posts: 18
Joined: 03 Oct 2012, 01:12
Location: Norway

Re: An alternative "changes" cheat code

Post#3 by Thor » 21 Oct 2012, 22:08

fanthom wrote:@Thor

i think i like it (must do some tests first) but would like to avoid creating another cheat as we have many already.
what about specifying xzm module as a destination in standard 'changes=' cheat?
example:
'changes=/mnt/sdb2/porteus/modules/changes.xzm'
if linuxrc find that destination is a module then everything is saved in the memory as in 'always fresh' mode.
(/porteus/modules is not obligatory, xzm could be saved anywhere but then users would have to load it with 'extramod=' or 'load=' cheats during next boot)
the path to the module could be exported to /mnt/live/tmp/changes file (i like /mnt/live as it's outside of aufs so any user damage made to it does not persist the reboot).

if rc.6 (during halt/shutdown) find /mnt/live/tmp/changes then:
a) unpacks the module
b) find and delete all files which are marked as .wh. in /mnt/live/memory/changes
c) copy all files (except .wh.) from /mnt/live/changes/{bin,etc,home,lib,lib64,opt,root,sbin,usr,var} to the unpacked module
d) create new module and replace the old one

as you know the only problem are big files and squashing them in the module would take a long time.
there is no direct solution for that (i would like to avoid hardcoding any size limit) but all big downloads in 99% of cases lands in $HOME.
we could advise users to use 'changes=something.xzm" plus optionally "magic folders" set on their home folder to solve this problem (magic folders create a layer above aufs so anything saved on them does not go to aufs rw branch)

pros:
a) Porteus with 'changes=' cheat is faster than ever before as everything is stored in the memory as in 'always fres' mode
b) nothing is is written on a device until shutdown (less I/O banging your flash drive)

cons:
a) user may lose whole session in case of power failure or porteus hang (both rare cases).
.
Yes it is the msquash time issue... I once did a similar script I called Br. Ring. One could chose what files to track changes for an do a module of. But that is too complicated for beginners since they would be clueless. But taking away "most meaningless changes" made it do the job pretty fast on my old system.... But even this small one here is like 20 times slower on my old pcs... I feel like 95% of the changes I wanna track are in my home folder. Those other 5 % are typically configuration of devices like sound. BUT those I only want to change one time and have them stay on that change. Hence, 1 "soundfix.xzm" is better then repeatingly do a mksquash...


fanthom wrote: i may have a solution on big files and long squashing/unsquashing problem:
'changes=' cheat would work as previously (no saving to xzm) but destination save.dat/folder/device would be inserted to aufs as read-only (same as module: =rr) while read-write (=rw) branch will be set on tmpfs.
this way rc.6 (or rather cleanup script from initrd after dismantling of /union) will have to do simple 'cp -a' operation as in reality destination file/folder/device is read-write :)
this way we wont have to waste resources on unsquash/squash: just dump all changes from RAM to a device at the very last stage of rebooting.
that should be fast enough even for large files.

need to do some testing if that works at all (i'm 99% sure it does), but new syntax would look like:
'changes=EXIT:/mnt/sdXY/some-folder'
so linuxrc will know that:
a) /mnt/sdXY/some-folder needs to be treated as a module (read only for aufs)
b) insert it as very last in the queue
c) set =rw branch in tmpfs (like in always fresh mode)

ofc without EXIT: string changes would be saved as normal (in the real time).

discussion is still opened so please share other ideas how to handle "changes on exit" task the best way we can.
This I like the sound of.. It was what I was searching for but couldn't find. And if I had found it, I would probably not written this script.
Old changes, read only..mm..should be quick and it reduces the new injected "changes" branch a lot. Could absolutely make system noticeable faster on older hardware... But the best part is new changes only stored at exit, since some old USB are inhumanly slow to write.

My vote goes to your latter idea as the way to go. And cleaning up that changes folder/file a bit by removing all those aufs spesifics the user only has to right click and convert to module to get a "total system backup". Very userfriendly and probably a lot faster and friendlier on the ramdisk size needed. Reinstallation...:Format, frugal and post that 200+mb changes squash module into the module folder :). VBox, ATI, settings and all is right back (in a cleaner way)...Actually. One could also add a conf file in some porteus folder where at any time one could remove and or add folders one wish to track for changes...So when all of hardware is working smoothly one could take away those folders that could be affecting devices if changed..

Ad hoc..One should also do what ive done in my own script now (0005.. maybe)...Give the user a short timeout during reboot, with a chance to bailout from saving changes to disk. Sometimes they are not what one had planned when one edited the bootline.

User avatar
fanthom
Moderator Team
Moderator Team
Posts: 5666
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland
Contact:

Re: An alternative "changes" cheat code

Post#4 by fanthom » 22 Oct 2012, 09:01

Hence, 1 "soundfix.xzm" is better then repeatingly do a mksquash...
agree.
i have same module for network/sound/ssh, etc settings. it's named as 0099 so it wont slow down booting with updating the caches.
magic_folders on /home and i'm good to go.
never had any troubles with this setup but as you have pointed already - it's rather for advanced users.
And cleaning up that changes folder/file a bit
changes=EXIT: will skip folders like /boot, /dev, /mnt, /media, /proc, /run, /sys and /tmp by default.
i would be careful with deleting anything from remaining folders (even /var) but will double check that.
One could also add a conf file in some porteus folder where at any time one could remove and or add folders one wish to track for changes.
let's try existing solution first without making things too much complicated.
if it wont be enough then i'll think about it (should be easy to implement).
Give the user a short timeout during reboot, with a chance to bailout from saving changes to disk.
hmmm.. but user would have to provide a path where changes would be stored, then cleanup would have to do rw/posix test, etc...
i dont want to make shutdown slower (after my last tweaks it's a breeze) but maybe will think about a "rescue" key which (when pressed) would drop down to the ash shell where user could do what he wants.
not sure if it's possible but will try.

thanks for suggestions.
Please add [Solved] to your thread title if the solution was found.

User avatar
Thor
Contributor
Contributor
Posts: 18
Joined: 03 Oct 2012, 01:12
Location: Norway

Re: An alternative "changes" cheat code

Post#5 by Thor » 22 Oct 2012, 10:24

fanthom wrote:
Give the user a short timeout during reboot, with a chance to bailout from saving changes to disk.
hmmm.. but user would have to provide a path where changes would be stored, then cleanup would have to do rw/posix test, etc...
i dont want to make shutdown slower (after my last tweaks it's a breeze) but maybe will think about a "rescue" key which (when pressed) would drop down to the ash shell where user could do what he wants.
not sure if it's possible but will try.

thanks for suggestions.
This is useful. I did it as uncomplicated as I could think of... echo "Timeout 5 seconds...Press any key to cancel the saving of your changes this session" && read 5 && exit

User avatar
fanthom
Moderator Team
Moderator Team
Posts: 5666
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland
Contact:

Re: An alternative "changes" cheat code

Post#6 by fanthom » 22 Oct 2012, 14:16

@Thor,

sorry i misunderstood and now i get it.
i hate delays (rebooting porteus constantly while developing) so will go with 5 sec delay but only if user used 'changes=EXIT:' cheat.
will also try to seek for 2 keys:
a) ENTER to start dumping changes on drive immediately
b) SPACE to skip
this way user wont have to wait for anything.

needs to do some extensive testing first but i'm getting convinced this could be default method of handling 'changes=' in Porteus-2.0.
will speak with other devs about it.
Please add [Solved] to your thread title if the solution was found.

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Re: An alternative "changes" cheat code

Post#7 by Rava » 03 Feb 2013, 10:33

Thor wrote:Automatically during shutdown changes In home-folders for guest and root with file size Less then approx 100kb are saved into a module and placed in the module folders where you loaded porteus from. Typically files of use under 100kb would be stuff like desktop-links, bookmarks, small scripts, desktop apaerance, language settings and so on..
But... that would also include cached file, like ones in $HOME/.mozilla/firefox/*.default/Cache , yes?

And me thinks no one needs any of these files, especially when one wants less writing to the device.

I checked my current FFx cache, and quite some files match the "less than 100kb" ...
Cheers!
Yours Rava

User avatar
fanthom
Moderator Team
Moderator Team
Posts: 5666
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland
Contact:

Re: An alternative "changes" cheat code

Post#8 by fanthom » 03 Feb 2013, 20:30

But... that would also include cached file, like ones in $HOME/.mozilla/firefox/*.default/Cache , yes?
And me thinks no one needs any of these files, especially when one wants less writing to the device.
user preferences differs and it's not possible to judge which files should be dumped on a device and which not.

there is a /etc/changes_exit.conf file which let's you decide which folders should be saved.
i could probably do a reverse and include there a folders which should never be saved but that would be complicated for a people who wants to save only $HOME/.mozilla/ as they would have to add every other folder from $HOME to /etc/changes_exit.conf.

for me an easier solution would be to disable caches in firefox preferences.
Please add [Solved] to your thread title if the solution was found.

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Re: An alternative "changes" cheat code

Post#9 by Rava » 03 Feb 2013, 22:29

fanthom wrote:for me an easier solution would be to disable caches in firefox preferences.
What about some special syntax in /etc/changes_exit.conf ?

Like, when the line / folder name starts with an "!" than that means, it and all its subfolders should not be saved?

Cause disabling caches in Firefox would change its performance, usually to the worse...
Cheers!
Yours Rava

User avatar
fanthom
Moderator Team
Moderator Team
Posts: 5666
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland
Contact:

Re: An alternative "changes" cheat code

Post#10 by fanthom » 03 Feb 2013, 22:46

What about some special syntax in /etc/changes_exit.conf ?
Like, when the line / folder name starts with an "!" than that means, it and all its subfolders should not be saved?
good thinking. will implement this for final.

thanks.
Please add [Solved] to your thread title if the solution was found.

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

Re: An alternative "changes" cheat code

Post#11 by Rava » 03 Feb 2013, 22:55

fanthom wrote:good thinking. will implement this for final.
yay! And please include some comment in the file about that syntax detail, maybe even using the guest's FFx Cache folder as default example?
(Since me thinks some other people will think that not saving some cache info on, say, SD cards or such media makes quite some sense. :) )
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Re: An alternative "changes" cheat code

Post#12 by Ed_P » 23 Feb 2013, 03:49

Is the changes=EXIT: in version 2.0?
Ed

User avatar
fanthom
Moderator Team
Moderator Team
Posts: 5666
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland
Contact:

Re: An alternative "changes" cheat code

Post#13 by fanthom » 23 Feb 2013, 10:04

Please add [Solved] to your thread title if the solution was found.

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Re: An alternative "changes" cheat code

Post#14 by Ed_P » 23 Feb 2013, 15:41

Ok. :)

But the format isn't mentioned in your link or boot/docs/cheatcode.txt so how do I use EXIT: and the name of the file?

Code: Select all

changes=EXIT:,porteussave.dat
??
Last edited by Ed_P on 23 Feb 2013, 15:45, edited 1 time in total.
Ed

User avatar
Hamza
Warlord
Warlord
Posts: 1908
Joined: 28 Dec 2010, 07:41
Distribution: Porteus
Location: France

Re: An alternative "changes" cheat code

Post#15 by Hamza » 23 Feb 2013, 15:43

Code: Select all

changes=EXIT:/path/to/save.dat
NjVFQzY2Rg==

Post Reply