FAA Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post here if you are a new Porteus member and you're looking for some help.
Post Reply
mysterylectricity
White ninja
White ninja
Posts: 17
Joined: 18 Feb 2015, 07:52
Distribution: 3.1
Location: San Francisco

FAA Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#1 by mysterylectricity » 24 Mar 2015, 21:49

Hi,

I'm running Porteus 3.1 with 4GB ram, 1GB devoted to win7u in Vbox, and an encrypted changes.dat file.

Often the system acts like it's swapping itself to death. Especially when switching between the VM and the KDE desktop, but often enough within one or the other when both are running (ie: problem will crop up at the desktop when Vbox is busy in the background or even mostly idle, and crop up in the Vbox even when the Porteus host is mostly idle), and apparent even when VBox isn't running ie: when I have too many tabs open in Firefox on KDE.

Oh: the symptom is long delays (10's of seconds) before mouse actions, closing windows, changing focus etc. are recognized and responded to (ie: redraws). Likewise display (both in VBox and KDE) sometimes freezes for long periods, ie: no performance meter updates.

This seems to happen regardless as to whether or not I have a small zram set-aside for swap, and regardless as to whether or not the zram swap is being used as reported by "free -m." Though I haven't done any benching or objective comparisons.

I'm doubly puzzled as to how the system can act like it's swapping even when there is no swap, but I can imagine what's going on:

I'm wondering if there's just too much or too little caching of one kind or another. I'm thinking important pages are being tossed to make room for disk buffers and read-ahead stuff of dubious value. In some cases I'm confused as to where data is being cached to and from, wondering if some programs manage caches of their own outside of the normal Linux/FS caches: ie: is Firefox and maybe even KDE maintaining caches of their own in on-disk files? I've got Firefox caching turned off in prefs, but that's not really the kind of caching I mean: more like caching display objects ie: tabs in Firefox, windows/contents in KDE.

I tend to remember there are some tunable parameters which might help, ie: which favor code pages over user data or something like that. But I forget what these parameters are and where they are set (was a BSD whiz 25 years ago, when just about everything was in rc.local: not up to speed on Linux/Porteus boot sequence).

What should I be looking for, and in what files should I modify my options?

Curious as to the limits of tuneability here, can I favor "out of memory" messages, particularly for user processes, over aggressive cache cleanup? That is, favor response time of previously open tabs in Firefox and disfavoring or flat out preventing the opening of new tabs until I've closed some old ones (bad habit).

Could even good ole' ulimit and/or sysctl be leveraged to help?

Thank you in advance for any assistance, I'll report back with any progress.

I think I'll try running from RAM, with EXIT in the changes cheatcode, etc. a bit more to see what impact that has.

-jeff

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#2 by francois » 25 Mar 2015, 02:08

Often the system acts like it's swapping itself to death.

I get the same behavior too, but on my desktop installation of porteus 3.1 on less performant laptop. Especially on my hp pavilion dv1540ca laptop and to a less extent on a msi 340x.

The use of 2 gig swap file helped, but did not completely solved the problem especially with more memory hungry packages like libreoffice and google-chrome:
https://forum.porteus.org/viewtopic.php?f=61&t=3499

kde is less performing than xfce, and the latter less than lxde.

For sure virtualbox porteus install will not help.

I have the impression that the previous editions of porteus were performing better, see the archives:
http://dl.porteus.org/i486/archive/

What type of linux box do you have? Can you give the specs?

It would be interesting that the devs comment on your comments and mine.
Voltaire: Le mieux est l'ennemi du bien.

mysterylectricity
White ninja
White ninja
Posts: 17
Joined: 18 Feb 2015, 07:52
Distribution: 3.1
Location: San Francisco

Re: Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#3 by mysterylectricity » 25 Mar 2015, 03:17

@francois:

I'm running on a laptop, sure enough. Intel Core i5 (2520? 520?) at 2.9x Mhz, dual cpu with hyperthreading. 4G (3.7xG available, Intel HD GFX et al probably eats some). Running Porteus 3.1 64-bit host, Win7Ultimate 32-bit guest.

I'm still convinced that tunables are worth exploring, especially in the face of behavior when VBox is not running.

But, that having been said:

At this very point in time, I'm noticing some quirky behaviors on the guest VM I've never noticed before on other win7 systems ie: update download and installation progress bars not working, totally ignoring auto update settings: installing updates willy-nilly: have seen similar behavior on other machines and not only that, but when I was litecoin mining I figured out how to solve with group policy etc. but never this bad: except to say that I noticed a friend's system also not tracking update progress correctly today. Coincidence? Also seeing a huge 151 MB win7 update, "System Update Readiness Tool for Windows 7", KB947821 dated Oct. 2014, which I've never noticed before but which seems to have implications far beyond what its innocuous title would imply. In particular maybe it will help with the progress bars, (kinda shotgunning/sidetracking in this post, throwing in every quirk in case someone hits on it with a web search and says, "Hey! I noticed that exact same thing today!")

And I'm thinking I've put VBox in the worst possible position by putting the VBox (.vhd?) file in an encrypted save.dat.

So.... at this point, I plan on creating a new VM:

1) In it's own, unencrypted file or partition.
2) 64 bit win7u guest on my 64 bit Porteus host.
--->3) Not installing "Guest Additions", at least not until all updates are applied and/or I have some
experience with the system w/o Guest Additions.<----
4) Scouring for reports on optimizations and bugs, esp. in VBox docs, of which I've only read a page or two as needed.

.... so I'm "sidetracked" into exploring VBox-specific issues right now, will report back on those while I'm rolling my finger tips waiting for info on cache behavior tuning in Porteus.

-jeff

mysterylectricity
White ninja
White ninja
Posts: 17
Joined: 18 Feb 2015, 07:52
Distribution: 3.1
Location: San Francisco

Re: Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#4 by mysterylectricity » 25 Mar 2015, 04:14

Oh: of more general relevance, I've trimmed my VM from 1GB to .5GB and my zram from ~600M to about 70M. I'm using the zram more as a crude indicator as to when traditional swapping actually starts, rather than as a practical swap file. This seems to have improved matters, but the system still seems too swappy: now, the delays are on the order of seconds and fractions as opposed to 10's of seconds and 10's of minutes: but on the other hand I've subconsciously modified my memory hungry behavior, as well. Next I'll try adding traditional swap as per the posts you referenced and report back. Curious as to the security risks of unencrypted swap from a technical perspective, but this is an issue addressed in forums dedicated to such matters.

Still think a lot can be done with cache/buffer tuning: in particular because after long touting the advantages of UNIX memory management over Windows (despite my long absence from UNIX/Linux), my experience coming back to Linux after 10 years of Windows is that Windows handles these situations better than Linux at this point.

Say it isn't so!

Of course, the desired behavior of a single-user OS and a multi-user OS will always be different. Although my Porteus install seems to become unusable from a practical standpoint (far?) more readily than under a similar workload under Win7, I can't say it's actually _crashed_ or hung under heavy load. But on the other hand, it's been a long time since I've had a win7 machine hang/crash simply due to a heavy workload, either. About the worst that's happened is having to wait a long time for the w7 system to respond to ctrl-alt-del, bring up the task manager, and kill the offending process.

I'll also try lighter desktops: maybe it's the desktop overhead that seems to be putting such a huge gap between expected and actual performance.

-jeff

jimwg
Shogun
Shogun
Posts: 322
Joined: 09 Oct 2013, 18:15
Distribution: porteus
Location: NYC

Re: Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#5 by jimwg » 09 Apr 2015, 17:30

Greetings!

Via user advice, I've never created SWAP in any of my 8gig Porteus (or Mint) flashes for years. I can't imagine what I'm missing since OpenOffice and SeaMonkey and GIMP snap into operation at a click and I save a lot of space. If it's a flash drive wear and tear prevention issue then I'd change, but otherwise what's the real advantage of using SWAP?

Jim in NYC

User avatar
francois
Contributor
Contributor
Posts: 4947
Joined: 28 Dec 2010, 14:25
Distribution: kde xfce porteus manjaro kubun
Location: Enfin l'été, le changement climatique attendu: le soleil.

Re: FAA Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#6 by francois » 10 Apr 2015, 01:21

The vbox expert for porteus is fanthom. :)
Voltaire: Le mieux est l'ennemi du bien.

rek769
White ninja
White ninja
Posts: 15
Joined: 22 Apr 2015, 14:55
Distribution: Porteus-v3.1 / Puppy Linux
Location: Big Blue

Re: FAA Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#7 by rek769 » 24 Apr 2015, 15:33

I was experiencing the same issue after having installed on an older USB I had not used for quite awhile.

Suggestions:
Be sure you are using a USB 2.0 flash drive, your computer has USB 2.0 compatibility, and that the port you are using is a 2.0 port. (of course 3.0 would be even better). My older Toshiba laptop has one 2.0 port and using that port with a confirmed 2.0 flash drive made a big difference for me.

Good luck

mysterylectricity
White ninja
White ninja
Posts: 17
Joined: 18 Feb 2015, 07:52
Distribution: 3.1
Location: San Francisco

Win7u32b in VBox Sleeps for Minutes Under Porteus on Flash

Post#8 by mysterylectricity » 26 Apr 2015, 08:35

Well, several things have happened since first posting.

The main thing is I've been focusing on using Porteus in production rather than ironing out every little mystery. Sorry :(

But this problem keeps coming and going like a Cheshire cat and it's driving me nuts.

Symptom summary: oftentimes Win 7 Ultimate 32bit seems to hang for long periods in VirtualBox under Porteus.

During these periods, Resource Monitor indicates a wide range of disk access times of up to 25 seconds. Often, if for example 30 or more
files are being written to, 10 or more files indicate access times of 10 seconds or more.

Only when the virtual machine is almost idle, when I see 10 or less processes reading or writing, do I get access times in the low milliseconds as expected.

Sometimes, the W7 window system (in the Vbox) becomes very unresponsive. This sometimes leads me to double click like mad out of frustration,
which of course only opens more processes and makes things worse.

Oftentimes, after a very nasty spell of these way-out file access times, the system will become responsive and usable again for quite a while.

I've upgraded my laptop from 4GB to 8GB, and subsequently have tried giving the VM 1GB through 3.7GB dedicated, and this has had little or no impact
on the problem.

On the long list of suspects at this point:

1)The W7u32b "Guest Additions" Package

2)One or more sub-optimal choices when choosing hardware or other options in VBox Manager before loading Windows.

3) One or more sub-optimal choices when changing hardware or other options in VBox Manager AFTER loading Windows: that is, by trying to fix the problem by choosing different options in VBM between reboots of the VM, I'm making things much worse.

4) Swap. Something bad is happening, maybe neither Windows nor Porteus understands the huge write time penalty of swapping to a flash drive. Especially
when writing in little chunks.

5) Memory ballooning, or some other VBox weirdness VBox docs are careful to warn about... and I'm just not reading or recalling the right section
of the VBox docs... shame on me.

6) My VBM settings under "root" and "guest" accounts are different and one works well and the other works poorly.

7) It has everything to do with write speed, especially small file write speed, of the flash drive involved.

8) New possibility: sometimes my system comes up in USB 2.0 mode, other times comes up or falls back to 1.0 or 1.1 mode.

One thing I've considered trying is giving the virtual box "native" control of it's own partition on the flash drive, as detailed in the VBox docs.

But... I'm not ready to go there yet. That would take a few hours, probably not help much, and giving Windows direct control of one of my partitions is
SCARY. One mistake and I have to reload all from backup, at least. Huge potential mess.

Now here is the odd thing. I've tried this out on three generations of flash drives:

1. A 32GB USB 3.0 adata with about 100 MB/sec large reads, maybe 20 MB/sec large writes, and a pathetic maybe 20 KB/sec small file write (I forget small file read).
2. A 120GB Visiontek USB 3.0 "SSD on a Stick" with at least 250MB/sec large reads, 250MB/sec large writes, and something like 10-20MB/sec small reads and writes.
3. Sandisk's newish Ultra Fit 64G USB 3.0 micro flash drive. At least 100MB/sec large writes, at least 100 MB sec large reads, and at least 1 MB/sec small reads and writes.

Now, actually, I'm running these all at USB 2.0 (or so I hope).

So performance I'm ACTUALLY getting is about 10-30 MB/Sec large reads and writes from all three drives.

I'm assuming, therefore, any perceived performance difference between the drives lies in that 20KB vs. 1MB vs. 10MB/sec small file write speed.

And sure enough I think I notice the whole system acting a lot peppier with the faster drives.

But NOT anything like an order of magnitude faster. And NOT with any consistency. When the VM goes haywire, the effect seems independent
of the type of flash drive.

It seems to me I was very happy with the mid-performance Sandisk, in particular, for at least a good week :)

I breathed a huge sigh of relief.

And then BLAM. I screwed something up again.

The idea that all three drives behave OK sometimes and really poorly at other times because my Porteus system is flipping the port back and forth between USB 1.x and 2.x is intriguing, and the theory is easily tested, but I bet it is a dead end.

Well, when and if I find the answers I'll be sure to share them.

But I'm at the point of tears here and about to flush the whole VBox thing, in favor of using some variant of VNC to log in to a real Windows machine from within Linux. At this point waiting .1 second for each screen redraw is winning HANDS DOWN over tearing my hair out for 3 minutes while the VM works through it's queue of "access time increases with the square of the overall MB/sec, multiplied by the cube of the number of pending disk ops" pileup before being responsive again.

Grrr. Oh well. Unfortunately I haven't been scientifically disciplined enough to narrow it down, or really to exclude ANY possibility.

Maybe 8 or 10 variables. I guess if i tried each one independently that's only 8 or 10 "trials".

But some trials have huge time penalty. Guess I'll start systematically with the easy ones.

But it seems I've mostly done that already and... little to show. Only more puzzlement.

I can only spend an hour a week on this one: my days as an IT pro are over, now I'm stuck with having
to use computers to solve other problems, rather that having the (luxury?) of computers BEING the main problem
to be solved.

Sometimes I miss those days :(

Other days I'm soooooo thankful for the change ha ha!

Thanks for the tips!

-Jeff

mysterylectricity
White ninja
White ninja
Posts: 17
Joined: 18 Feb 2015, 07:52
Distribution: 3.1
Location: San Francisco

Re: FAA Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#9 by mysterylectricity » 26 Apr 2015, 08:42

Hmm, I'd like to change the topic of this thread to:

"Win7u32b in VBox Sleeps for Minutes Under Porteus from Flash"

..so I'll spend a few minutes seeing if I can do that myself.....

-Jeff

User avatar
fanthom
Site Admin
Site Admin
Posts: 4567
Joined: 28 Dec 2010, 02:42
Distribution: Porteus Kiosk
Location: Poland, currently - Cork, IE
Contact:

Re: FAA Porteus acting too swappy/sleepy: (Cache?) Tunables?

Post#10 by fanthom » 26 Apr 2015, 21:02

@Jeff,
have you tried the same setup under any other Linux? just to make sure that its Porteus to blame and not any other component ...
Please add [Solved] to your thread title if the solution was found.

Post Reply