What Makes save.dat File Grow? There A "Prune" App For It?

[Ed_P]

please run 'logrotate /etc/logrotate.conf' manually and check if the job is done.

FYI

Code: Select all

root@porteus:/home/guest# ls -al /var/log
total 4192
drwxr-xr-x 31 root root      340 Jan 21 12:12 ./
drwxr-xr-x 32 root root      160 Jan 21 11:24 ../
drwxr-xr-x  2 root root       60 Jan 21 11:24 ConsoleKit/
-rw-r--r--  1 root root   389488 Jan 21 13:57 Xorg.0.log
-rw-r--r--  1 root root   177296 Jan 21 12:12 Xorg.0.log.old
-rw-r--r--  1 root root    42764 Dec 30 01:48 Xorg.1.log
-rw-r--r--  1 root guest   42764 Dec 29 02:30 Xorg.1.log.old
-rw-------  1 root root        0 May 31  2013 btmp
-rw-r--r--  1 root root     4828 Jan 21 11:24 cron
-rw-r--r--  1 root root   238898 Jan 21 12:40 debug
-rw-r--r--  1 root root    57641 Jan 18 12:49 debug.1.gz
-rw-r--r--  1 root root    43216 Jan 21 11:24 dmesg
-rw-r--r--  1 root root        0 Sep 13  2012 faillog
-rw-r--r--  1 root root       11 Jan 19 02:22 internet
-rw-r--r--  1 root root   292292 Dec 30 00:44 lastlog
-rw-r--r--  1 root root        0 Oct 20  2010 maillog
-rw-r--r--  1 root root  1111725 Jan 21 13:52 messages
-rw-r--r--  1 root root   461861 Jan 18 12:50 messages.1.gz
drwxr-xr-x  2 root root        3 May 16  2001 nfsd/
drwxr-xr-x  2 root root      113 Jul 19  2013 packages/
-rw-r--r--  1 root root     2050 Jan 21 12:40 pm-powersave.log
-rw-r--r--  1 root root     6583 Jan 21 12:40 pm-suspend.log
-rw-r--r--  1 root root      847 Jan 21 11:24 porteus-livedbg
drwxr-xr-x  2 root root        3 Jan  4  2012 removed_packages/
drwxr-xr-x  2 root root        3 Jan  4  2012 removed_scripts/
drwxr-xr-x  2 root root        3 May  3  2012 scripts/
-rw-r--r--  1 root root    37671 Jan 21 12:29 secure
drwxr-xr-x  4 root root       26 Jan  4  2012 setup/
-rw-r--r--  1 root root     3359 Jan 21 12:12 slim.log
-rw-r--r--  1 root root        0 Oct 20  2010 spooler
-rw-r--r--  1 root root   636557 Jan 21 13:57 syslog
-rw-r--r--  1 root root    31712 Jan 21 12:40 wpa_supplicant.log
-rw-r--r--  1 root root   660864 Jan 21 11:24 wtmp
root@porteus:/home/guest# logrotate /etc/logrotate.conf
error: /etc/logrotate.conf:31 duplicate log entry for /var/log/wtmp  <---- !! ??
root@porteus:/home/guest# ls -al /var/log
total 1608
drwxr-xr-x 31 root root     420 Jan 21 13:59 ./
drwxr-xr-x 32 root root     160 Jan 21 11:24 ../
drwxr-xr-x  2 root root      60 Jan 21 11:24 ConsoleKit/
-rw-r--r--  1 root root  396370 Jan 21 13:59 Xorg.0.log
-rw-r--r--  1 root root  177296 Jan 21 12:12 Xorg.0.log.old
-rw-r--r--  1 root root   42764 Dec 30 01:48 Xorg.1.log
-rw-r--r--  1 root guest  42764 Dec 29 02:30 Xorg.1.log.old
-rw-------  1 root root       0 May 31  2013 btmp
-rw-r--r--  1 root root    4828 Jan 21 11:24 cron
-rw-r--r--  1 root root  238898 Jan 21 12:40 debug
-rw-r--r--  1 root root   57641 Jan 18 12:49 debug.1.gz
-rw-r--r--  1 root root   43216 Jan 21 11:24 dmesg
-rw-r--r--  1 root root       0 Sep 13  2012 faillog
-rw-r--r--  1 root root      11 Jan 19 02:22 internet
-rw-r--r--  1 root root  292292 Dec 30 00:44 lastlog
-rw-r--r--  1 root root       0 Oct 20  2010 maillog
-rw-r--r--  1 root root      48 Jan 21 13:59 messages
-rw-r--r--  1 root root  173827 Jan 21 13:52 messages.1.gz
drwxr-xr-x  2 root root       3 May 16  2001 nfsd/
drwxr-xr-x  2 root root     113 Jul 19  2013 packages/
-rw-r--r--  1 root root    2050 Jan 21 12:40 pm-powersave.log
-rw-r--r--  1 root root    6583 Jan 21 12:40 pm-suspend.log
-rw-r--r--  1 root root     847 Jan 21 11:24 porteus-livedbg
drwxr-xr-x  2 root root       3 Jan  4  2012 removed_packages/
drwxr-xr-x  2 root root       3 Jan  4  2012 removed_scripts/
drwxr-xr-x  2 root root       3 May  3  2012 scripts/
-rw-r--r--  1 root root   37671 Jan 21 12:29 secure
drwxr-xr-x  4 root root      26 Jan  4  2012 setup/
-rw-r--r--  1 root root    3359 Jan 21 12:12 slim.log
-rw-r--r--  1 root root       0 Oct 20  2010 spooler
-rw-r--r--  1 root root       0 Jan 21 13:59 syslog
-rw-r--r--  1 root root   56534 Jan 21 13:57 syslog.1.gz
-rw-------  1 root root       0 Jan 21 13:59 wpa_supplicant.log
-rw-r--r--  1 root root     561 Jan 21 12:40 wpa_supplicant.log.1.gz
-rw-r--r--  1 root root       0 Jan 21 13:59 wtmp
-rw-r--r--  1 root root   19479 Jan 21 11:24 wtmp.1.gz
root@porteus:/home/guest# 

[Ed_P]

I manually run logrotate prior to shutting down and it seems that the purges aren't really purges, it appears that the older log entries are simply compressed to other files, .gz and .old. Can these files be deleted?

Code: Select all

guest@porteus:~$ logrotate.sh
total 1544
drwxr-xr-x 37 root root     320 Mar 13 15:16 .
drwxr-xr-x 38 root root     160 Mar 13 15:16 ..
drwxr-xr-x  2 root root      60 Mar 13 15:16 ConsoleKit
-rw-r--r--  1 root root   42017 Mar 13 15:16 Xorg.0.log
-rw-r--r--  1 root root   42734 Mar 13 15:14 Xorg.0.log.old
-rw-r--r--  1 root root   42764 Dec 30 01:48 Xorg.1.log
-rw-r--r--  1 root guest  42764 Dec 29 02:30 Xorg.1.log.old
-rw-------  1 root root       0 May 31  2013 btmp
-rw-r--r--  1 root root    7358 Mar 13 15:16 cron
-rw-r--r--  1 root root   40172 Mar 13 15:16 debug
-rw-r--r--  1 root root   27291 Mar 13 02:06 debug.1.gz
-rw-r--r--  1 root root   43284 Mar 13 15:16 dmesg
-rw-r--r--  1 root root       0 Sep 13  2012 faillog
-rw-r--r--  1 root root      11 Mar 10 14:45 internet
-rw-r--r--  1 root root  292292 Dec 30 00:44 lastlog
-rw-r--r--  1 root root       0 Oct 20  2010 maillog
-rw-r--r--  1 root root  120183 Mar 13 15:17 messages
-rw-r--r--  1 root root   88646 Mar 13 02:37 messages.1.gz
drwxr-xr-x  2 root root       3 May 16  2001 nfsd
drwxr-xr-x  2 root root      47 Jan 16  2013 packages
-rw-r--r--  1 root root    2094 Mar 13 15:16 pm-powersave.log
-rw-r--r--  1 root root    6582 Mar 13 01:26 pm-suspend.log
-rw-r--r--  1 root root    1003 Mar 13 15:16 porteus-livedbg
drwxr-xr-x  2 root root       3 Jan  4  2012 removed_packages
drwxr-xr-x  2 root root       3 Jan  4  2012 removed_scripts
drwxr-xr-x  2 root root      47 Jan 16  2013 scripts
-rw-r--r--  1 root root   54835 Mar 13 15:21 secure
drwxr-xr-x  4 root root      26 Jan  4  2012 setup
-rw-r--r--  1 root root    4823 Mar 13 15:16 slim.log
-rw-r--r--  1 root root       0 Oct 20  2010 spooler
-rw-r--r--  1 root root  250768 Mar 13 15:20 syslog
-rw-r--r--  1 root root   56534 Jan 21 13:57 syslog.1.gz
-rw-------  1 root root   15232 Mar 13 15:16 wpa_supplicant.log
-rw-r--r--  1 root root     561 Jan 21 12:40 wpa_supplicant.log.1.gz
-rw-r--r--  1 root root  331776 Mar 13 15:16 wtmp
-rw-r--r--  1 root root   19479 Jan 21 11:24 wtmp.1.gz
error: /etc/logrotate.conf:31 duplicate log entry for /var/log/wtmp
total 1212
drwxr-xr-x 37 root root     340 Mar 13 15:21 .
drwxr-xr-x 38 root root     160 Mar 13 15:16 ..
drwxr-xr-x  2 root root      60 Mar 13 15:16 ConsoleKit
-rw-r--r--  1 root root   42017 Mar 13 15:16 Xorg.0.log
-rw-r--r--  1 root root   42734 Mar 13 15:14 Xorg.0.log.old
-rw-r--r--  1 root root   42764 Dec 30 01:48 Xorg.1.log
-rw-r--r--  1 root guest  42764 Dec 29 02:30 Xorg.1.log.old
-rw-------  1 root root       0 May 31  2013 btmp
-rw-r--r--  1 root root    7358 Mar 13 15:16 cron
-rw-r--r--  1 root root   40172 Mar 13 15:16 debug
-rw-r--r--  1 root root   27291 Mar 13 02:06 debug.1.gz
-rw-r--r--  1 root root   43284 Mar 13 15:16 dmesg
-rw-r--r--  1 root root       0 Sep 13  2012 faillog
-rw-r--r--  1 root root      11 Mar 10 14:45 internet
-rw-r--r--  1 root root  292292 Dec 30 00:44 lastlog
-rw-r--r--  1 root root       0 Oct 20  2010 maillog
-rw-r--r--  1 root root  120231 Mar 13 15:21 messages
-rw-r--r--  1 root root   88646 Mar 13 02:37 messages.1.gz
drwxr-xr-x  2 root root       3 May 16  2001 nfsd
drwxr-xr-x  2 root root      47 Jan 16  2013 packages
-rw-r--r--  1 root root    2094 Mar 13 15:16 pm-powersave.log
-rw-r--r--  1 root root    6582 Mar 13 01:26 pm-suspend.log
-rw-r--r--  1 root root    1003 Mar 13 15:16 porteus-livedbg
drwxr-xr-x  2 root root       3 Jan  4  2012 removed_packages
drwxr-xr-x  2 root root       3 Jan  4  2012 removed_scripts
drwxr-xr-x  2 root root      47 Jan 16  2013 scripts
-rw-r--r--  1 root root   54835 Mar 13 15:21 secure
drwxr-xr-x  4 root root      26 Jan  4  2012 setup
-rw-r--r--  1 root root    4823 Mar 13 15:16 slim.log
-rw-r--r--  1 root root       0 Oct 20  2010 spooler
-rw-r--r--  1 root root  250768 Mar 13 15:20 syslog
-rw-r--r--  1 root root   56534 Jan 21 13:57 syslog.1.gz
-rw-------  1 root root   15232 Mar 13 15:16 wpa_supplicant.log
-rw-r--r--  1 root root     561 Jan 21 12:40 wpa_supplicant.log.1.gz
-rw-r--r--  1 root root       0 Mar 13 15:21 wtmp
-rw-r--r--  1 root root    9965 Mar 13 15:16 wtmp.1.gz

And my logrotate.sh is simply:

Code: Select all

#!/bin/sh

if [ `whoami` != "root" ]; then
  ktsuss "$0"
  exit
fi

ls -al /var/log
logrotate /etc/logrotate.conf
ls -al /var/log

read

[dustbunnies]

(mounting and) pruning an already existing savefile would be the wrong approach, eh.
Building a list of ignored/excluded paths and performing (scripted) housecleaning prior to updating the savefile, is a good approach... but deciding what to exclude or not, by default, is a tough call. Each user probably needs to review, and tailor, their list.

example:
I purge browser cache prior to updating the savefile, but someone accesing the net through a slow, or metered, connection might prefer to retain the cached content vs re-downloading later.

I recommend it run when booting

Yep, I agree. I would also autorun updatedb at each boot, but I've heard people complain that adds 4.0056sec (or whatever) startup delay... so I recommend calling updatedb immediately prior to each savefile update. cleanup }} updatedb }} savefile-update

[fanthom]

I recommend calling updatedb immediately prior to each savefile update. cleanup }} updatedb }} savefile-update
we have something along these lines already:
changes=EXIT: cheatcode plus edit /etc/changes-exit.conf

[Ed_P]

I recommend calling updatedb immediately prior to each savefile update. cleanup }} updatedb }} savefile-update

??

Code: Select all

guest@porteus:~$ su
Password: 
root@porteus:/home/guest# cleanup
bash: cleanup: command not found
root@porteus:/home/guest# updatedb
bash: updatedb: command not found
root@porteus:/home/guest# 

FWIW The only purpose I use the save,dat function is to save my configuration settings and changes when I shutdown Porteus. If I download, or create, a file I save it to my system's hard drive which is NTFS format. Saving the file to my guest folder than forcing a save.dat update seems convoluted and I doubt what save.dat was designed for. Am I wrong?

changes=EXIT: cheatcode plus edit /etc/changes-exit.conf

Interesting. Thanks.

[dustbunnies]

@Ed, re: "cleanup"
fanthom was quoting my post, and I was speaking hypothetically (vs literally referring to an existing "cleanup" command)

we have something along these lines already:
changes=EXIT: cheatcode plus edit /etc/changes-exit.conf

Sigh.
We're in a suggestions thread.
"cheatcode plus edit mumbletygook" does not fulfill OP's There A "Prune" App For It?

Re: Timing Your Own Auto-Save To Flash
http://forum.porteus.org/viewtopic.php? ... 42&p=21342
Postby brokenman » 17 Jan 2014, 11:59

From the cheatcodes.txt file:

While you are running a session with changes=EXIT, you can use the
'dump-session' command (run as root from the command line) to move all
of your changes from RAM to your storage media. This will reduce the
risk of data loss and free up your RAM.

So you may use whatever method you prefer to run the command dump-session at your chosen interval.

For me, the result of folowing that instruction was:
-- no changes saved
-- state was instantly reverted to the state in effect at the start of the current xsession

I attempted to workaround by outcommenting (from memory) 3 or 4 lines near the bottom of the dump-session script.
No joy.

fanthom's post #3 earlier in this current thread:

i have the impression you are really trying hard to break something.
maybe you could link /mnt/live/memory/changes to /porteus/anydata? mind that you would have to remount aufs with 'udba' flag when modifying rw branch directly. please have a look on 'dump-session' script.

It's like THIS:

Since 2009, across most puppy variants (Distrowatch #9 currently) by convention, if a savefile exists:
a bullseye icon is displayed to the desktop, enabling on-demand, at will, ability to backup/update the savefile

Since 2010 (IIRC the date correctly) antix linux (Distrowatch #41 currently) supports on-demand savefile update
if user has elected "semi-automatic" persistence mode, but without conditionally-displayed desktop icon.
Instead, user must: menu}antixCC}System}Persistence}SavePersistence:

Both of the above, in conjunction with on-demand availability during a live session,
will also prompt user at shutdown, asking whether pre-shutdown savefile update is desired.
(some puppies, and some antix config modes, perform the shutdown save without asking, though)

Porteus isn't meeting user need/expectation in this regard and replies like
"udbag and a side order of fries"
and
"Are you trying to BREAK something?"
suggest that despite repeated requests, pleas, suggestions... there's still a user/dev disconnect.

magic folders and roocopy and whatnot provide great flexibility... but in thread after thread, isn't it apparent that new users
are being overwhelmed by all the choices while they're still just learning how to walk?

In addition to the two porteus forum threads I cited above, there are numerous other posts indicating that
toram and/or on-demand save (safety waypoint) are desired usage parameters.

save session
http://forum.porteus.org/viewtopic.php?f=81&t=3016
Postby rych » 08 Jan 2014, 22:47
Does my session live completely in the RAM? How often are the changes saved to the persistent storage (/porteus/changes on my ext4 USB drive, I guess)? Can I force it? Can I change the frequency? Or is it saved on Logout only?

http://forum.porteus.org/viewtopic.php?f=53&t=3102
I suspect that my USB flash drive is accessed too frequently, even though I have a 4GB RAM and have put Firefox cache folder on the host system HDD. I'd like to investigate

[fanthom]

Saving the file to my guest folder than forcing a save.dat update seems convoluted and I doubt what save.dat was designed for.
save.file was designed for providing persistence on non posix compatible filesystems and it doesn't matter how you achieve this goal (saving in real time, saving on demand, using for magic folders, even copying single files manually).

"cheatcode plus edit mumbletygook" does not fulfill OP's There A "Prune" App For It?
well - it actually does. it's even better as you dot need any 'prune app' or script at all. just edit changes-exit.conf and select what you want to save, no need to prune/delete/remove anything. shutdown still takes 3 secs and this is what we want.

For me, the result of folowing that instruction was:
-- no changes saved
i guess you never meet required condition:
While you are running a session with changes=EXIT, you can use the 'dump-session' command

Since 2009, across most puppy variants (Distrowatch #9 currently) by convention, if a savefile exists:
a bullseye icon is displayed to the desktop, enabling on-demand, at will, ability to backup/update the savefile
we could do the same and create a .desktop file which would run 'dump-session' scritp. probably matter of preferences but i like to have clean desktop (no offense but last time i booted puppy some time ago it's desktop looked bit messy, maybe something changed - dunno) so i would leave it up to the users.

Both of the above, in conjunction with on-demand availability during a live session,
will also prompt user at shutdown, asking whether pre-shutdown savefile update is desired.
we have this option as well when user boots with 'changes=EXIT:'

despite repeated requests, pleas, suggestions... there's still a user/dev disconnect.
i'm getting annoyed by all these people coming here and trying to convert porteus to puppy with all the popups/wizards/etc...
from my point of view such system is a pain to use and i would probably have to abandon it.

our approach is different:
- clean desktop
- clean session
- no flashy popup windows/tips/wizards
- fast boot and shutdown
- boot the OS -> do your job -> turn of the PC without unnecessary clicks
- if something is not clear then spend 5 mins on reading docs and set the system according to your preferences
key points are highlighted.

porteus will never be a 'puppy like' system even by the cost of loosing a 'newbie friendly' stamp.
sorry to disappoint.

[Ed_P]

Saving the file to my guest folder than forcing a save.dat update seems convoluted and I doubt what save.dat was designed for.
save.file was designed for providing persistence on non posix compatible filesystems and it doesn't matter how you achieve this goal (saving in real time, saving on demand, using for magic folders, even copying single files manually).

Oh. Ok, thank you.

Given it's more significant importance may I suggest a script be included that creates a desktop icon that supports doing a manual backup to the save.dat file, if it exists? Thus if someone wants it, or feels a need for it, they can execute the script and have a desktop icon for manually updating the save.dat file. Seem reasonable?

[fanthom]

do we really need a script for this?

grab any .desktoop file from /usr/share/applications and place on your desktop then edit EXEC line like here:

Code: Select all

Exec=/opt/porteus-scripts/xorg/psu dump-session

[Ed_P]

do we really need a script for this?

Good grief!! YES

Why? Because:

a. Not everyone knows about this /usr/share/applications folder, me being the 1st one. (An interesting surprise I might add.)
b. Modifying a desktop file requires more than changing the Exec parm.
b1. You need to rename the file
b2. You need to change the Icon parm.
b2a. You need to find an icon to assign to the Icon parm.
b3. You need to change the Name parm.
b4. And for I know there may need to be other parms changed as well. StartupNotify, Categories, etc.

Maybe a script isn't the best option. Maybe creating a desktop file that updates the save.dat file and storing it in the /usr/share/applications folder and distributing it with Porteus is a better option. If someone wants the function they can copy it to their /home/guest/Desktop folder.

As an interim you could post the source for the desktop file and the command(s) to make it a functioning desktop file and people who want it could try it.

[brokenman]

Code: Select all

[Desktop Entry]
Name=Session saver
comment=Save you current session
GenericName=Save session
Exec=/opt/porteus-scripts/xorg/psu dump-session
Icon=gtk-save
Terminal=false
StartupNotify=true
Type=Application
Categories=System;Utility;

Untested but placing this file on your desktop should do what you want. So what you want is a desktop icon/menu item that you can click which will dump your current session to a save.dat file or a module?

[Ed_P]

Untested but placing this file on your desktop should do what you want. So what you want is a desktop icon/menu item that you can click which will dump your current session to a save.dat file or a module?

It seems from other posts that people want it to dump to a save.dat file. How would saving to a module be done? And what would the advantage be?

BTW and FWIW I sometimes copy my save.dat file to a save.dat.xzm file which I include when I boot to extramod mode. (which is what I boot to when I want to test something or when the grandkids are here. )

[brokenman]

How would saving to a module be done?
Saving to a module is done by issuing: dir2xzm /mnt/live/memory/changes /path/to/module.xzm. Some folders do not need to be saved so a script to prune what is not needed would be used.

And what would the advantage be?
Dumping to save.dat is only relevant if using FAT32. It can only be used when booting with changes=EXIT cheatcode. Saving to a module could be used in always fresh mode and on any file system.

[Ed_P]

Thank you brokenman. Very informative.

[brokenman]

I should add that when saving the changes as a module you can NOT create the module within the live system (E.g /home/guest/module.xzm or /tmp/module.xzm) because you would be caught in a loop where the changes are always changing because you are creating a module there.