zram for extension of usable space for aufs

[Quax]

Hi all,

long time no post

Preparing the next FluxFlux version I have integrated zram (successor of compcache) in linuxrc.

zram explained:

...creates a RAM based block device which acts as a swap disk, but is compressed and stored in memory instead of swap disk (which is slow), allowing very fast I/O and increasing the amount of memory available before the system starts swapping to disk.

Due to the fact that most netbooks today are memory constrained but have enough power to handle (de)compression
using zram is a perfect way ro increase the available virtual diskspace in live mode while using aufs.

On my test system (Atom Z520 with 2GB Ram) I could increase the ramsize in linuxrc from 60% to 80% while preserving
the same amount of available RAM while gaining 400MB more free diskspace. 8)

For those interested in here are the relevant parts from linuxrc/liblinuxlive.

liblinuxlive:

Code: Select all

...
# ==================================================================
# RAM compressing/optimizing function#s
# Author: Manfred Mueller, http://fluxflux.net
#
# Heavily based on zramswap-enabler from Sergey "Shnatsel" Davidoff,
# https://launchpad.net/~shnatsel
# ==================================================================

zram_start() {

   # get the number of CPUs
   num_cpus=$(cat /proc/cpuinfo | grep "cpu cores" | cut -d: -f2 | sed 's/ //g' | uniq)

   # set decremented number of CPUs
   decr_num_cpus=$(( $num_cpus - 1 ))

   # get the amount of memory in the machine
   mem_total_kb=$(grep MemTotal /proc/meminfo | rev | cut -d" " -f2 | rev)
   mem_total=$((mem_total_kb * 1024))

#   # determine modprobe parameter name, depending on kernel version
#   if (uname --release | grep -E --silent "^3\.[123]"); then
#   num_devices_parameter='zram_num_devices'
#   else
#   num_devices_parameter='num_devices'
#   fi

   # load zram modules
   modprobe zram num_devices=$num_cpus
	
   # initialize the devices, set total size to 1/2 RAM
   for i in $(seq 0 $decr_num_cpus); do
   echo $((mem_total / 2 / num_cpus)) > /sys/block/zram$i/disksize
   done
	
   # Creating swap filesystems
   for i in $(seq 0 $decr_num_cpus); do
   mkswap -L zram$i /dev/zram$i
   done

   # Switch the swaps on
   for i in $(seq 0 $decr_num_cpus); do
   swapon -p 100 /dev/zram$i
   done
}

zram_stop() {

   # Switching off swap
   for i in $(grep -E --only-matching '/dev/zram[[:digit:]]+' /proc/swaps); do
   swapoff $i
   done

   # remove module, wait instead of failing if it's busy
   rmmod --wait zram
}
...

linuxrc:

Code: Select all

...
# zram
if [ "$(cmdline_parameter zram)" != "" ]; then
if [ ! "$GERMAN_IS_ENABLED" ]; then
echolog "Using zram to increase usable RAM..."
else
echolog "Benutze zram zur Vergrößerung des nutzbaren Arbeitsspeichers..."
fi
fi

zram_start
...

RAMSIZE=$(cmdline_value ramsize)
if [ "$RAMSIZE" = "" ]; then 
   if [ ! "$(cmdline_parameter zram)" = "" ]; then
	    RAMSIZE="80%"
   else
	    RAMSIZE="60%"
   fi
fi
...

cleanup:

Code: Select all

...
# kill all processes which can be killed
killall_unneeded $(pidof cleanup)

# zram
if [ "$(cmdline_parameter zram)" != ""]; then
if [ ! "$GERMAN_IS_ENABLED" ]; then
echolog "disabling zram..."
else
echolog "Deaktiviere zram..."
fi
fi

zram_stop

...

Regards,

Quax

[fanthom]

hi Quax,

funny thing - just today i started reading about zram and playing with it.
unfortunately my test didn't go far as seems that zram/zsmalloc is broken in 3.4.1 kernel:

Code: Select all

root@porteus:/home/guest/work# modprobe zram 
FATAL: Error inserting zram (/lib/modules/3.4.1-porteus/kernel/drivers/staging/zram/zram.ko): Unknown symbol in module, or unknown parameter (see dmesg)
root@porteus:/home/guest/work# dmesg | tail
[ 3803.031802] zsmalloc: module is from the staging directory, the quality is unknown, you have been warned.
[ 3803.031807] zsmalloc: module license 'unspecified' taints kernel.
[ 3803.031809] Disabling lock debugging due to kernel taint
[ 3803.031907] zsmalloc: Unknown symbol alloc_vm_area (err 0)
[ 3803.031925] zsmalloc: Unknown symbol free_vm_area (err 0)
[ 3803.031937] zsmalloc: Unknown symbol __supported_pte_mask (err 0)
root@porteus:/home/guest/work#

3.3.x kernel works ok but i'll probably go with 3.4.x for porteus-1.2 (also - zram is still in staging so i dont trust it fully).

anyway - my thoughts about zram (without any testing):
a) should be enabled only when user is not using 'changes=' cheat
b) should be enabled only when PC has 1GB or less of available RAM
reason
a) we need CPU power to decompress data from modules
b) porteus supports encryption which may cause extra CPU charge

i'll keep an eye on zram feature.

as always - thanks for sharing

Posted after 43 minutes 37 seconds:
i see now that you have made zram optional through 'zram' cheatcode.
this is a good approach but i would drop calculations and made it more flexible by letting users decide about number of devices and their size.
something like "zram=2:300M"
where "2" would be a number of devices and "300M" a size for each.

Cheers

EDIT:\\
kernel issue solved - zram depends on zcache which can be enabled only if you compile CRYPTO as 'y' and not 'm' - ehhhh....

[brokenman]

Nice bleeding edge work! I guess this is perfect for those small factor devices, and other portable devices that will no doubt become more popular with time. Personally i like my computers big and my women small.

[Quax]

@brokenman:

This is totally OT but some members depending on their member category maybe feeling offended by your signature...

How do i become super user?
Wear your underpants on the outside and put on a cape ...

Porteus admins should definitely think about changing the star beneath to a figure wearing a cape and underpants on the outside...

Quax