[HOWTO] Tweaking mc's user menu - folder compression

Post tutorials, HOWTO's and other useful resources here.
User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

[HOWTO] Tweaking mc's user menu - folder compression

Post#1 by Rava » 01 Dec 2021, 21:28

Thanks to ncmprhnsbl for the head up on /etc/mc/mc.menu

Trying to set up multi-core support for mc's folder compression user menu ability, I was looking into the options of the compression programs used - gzip (.gz), bzip2 (.bz2), 7za (.7z), xz (.xz), zstd (.zst).

Some are threadable, others not.

Looking into it I found options some of you might find useful, and it is easy to tweak the mc user menu into using such options.

E.g. gzip - could be you find this option

Code: Select all

--rsyncable   make rsync-friendly archive
useful.
Easily to implement. Just change the original line from /etc/mc/mc.menu from

Code: Select all

        tar cf - "$Pwd" | gzip -f9 > "$tar.tar.gz" && \
into

Code: Select all

        tar cf - "$Pwd" | gzip -f9 --rsyncable > "$tar.tar.gz" && \
__________________________________________

Back to my reason for wanting to tweak the menu entries - the ability of some compression programs for threadability.

gzip and bzip2 have no threadability. (At least the version shipped with Porteus do not) xz, 7za and zstd have threadability.

Cave! When tweaking the setting into using all available cores and you compress a large folder (as in: huge data) it could slow down your running system during compression.
__________________________________________

E.g. xz : -T [n] or --thread=[NUM] with 0 being auto maximum

Better not use the -T 0 option with xz.
When you have a 4 core, set -T 2 or -T 3, but not -T 0. When you mostly compress folders with small and few files, it would not matter.
But when you forgot you set the xz option to 0 = auto maximum - and then compress a folder with huge data - your system could appear to be frozen, and an unusable system, even only for half a minute, is often less desirable than having to wait a bit longer for the mc folder compression to be finished.

To be on the safe side of things, when you have a 4 core CPU system, you could tweak the mc user menu into using 3 cores for folder compression using xz by changing the original line from /etc/mc/mc.menu from

Code: Select all

        tar cf - "$Pwd" | xz -f > "$tar.tar.xz" && \
into

Code: Select all

        tar cf - "$Pwd" | xz -f -T 3 > "$tar.tar.xz" && \
__________________________________________

7za: -mmt[N] : set number of CPU threads

So, by this syntax the tweak should be this: change

Code: Select all

        tar cf - "$Pwd" | 7za a -si "$tar.tar.7z" && \
into

Code: Select all

        tar cf - "$Pwd" | 7za a -si -mmt3 "$tar.tar.7z" && \
__________________________________________

zstd: -T# : spawns # compression threads (default: 1, 0==# cores)

Finally, by that syntax tweak should be this: change

Code: Select all

        tar cf - "$Pwd" | zstd -f > "$tar.tar.zst" && \
into

Code: Select all

        tar cf - "$Pwd" | zstd -f -T3 > "$tar.tar.zst" && \
Cheers!
Yours Rava

User avatar
M. Eerie
Moderator
Moderator
Posts: 620
Joined: 31 Aug 2017, 21:18
Distribution: Nemesis Xfce/MATE x64

[HOWTO] Tweaking mc's user menu - folder compression

Post#2 by M. Eerie » 02 Dec 2021, 19:27

There's also lbzip2, pbzip2 and pigz . All of them use parallel compression.

I've been using pigz for years, but I like your approach.
> Does not compute_ 🖖

https://forum.porteus.org/viewtopic.php?p=94310#p94310
https://forum.porteus.org/viewtopic.php?p=84002#p84002
https://forum.porteus.org/viewtopic.php?p=77174#p77174
https://forum.porteus.org/viewtopic.php?f=39&t=8584

User avatar
Rava
Contributor
Contributor
Posts: 5401
Joined: 11 Jan 2011, 02:46
Distribution: XFCE 5.01 x86_64 + 4.0 i586
Location: Forests of Germany

[HOWTO] Tweaking mc's user menu - folder compression

Post#3 by Rava » 03 Dec 2021, 13:12

M. Eerie wrote:
02 Dec 2021, 19:27
There's also lbzip2, pbzip2 and pigz . All of them use parallel compression.
All are new to me. Thanks for the heads up, i will all check out. And all can be easily integrated in mc's user menu.

As first example the redirect into file is used, and the next example no redirect into file.
You have to figure out which approach the program you want to add to the mc's user menu supports.

The gz code uses redirect into file - and so do bz2 and xz:

Code: Select all

3       Compress the current subdirectory (tar.gz)
        Pwd=`basename %d /`
        echo -n "Name of the compressed file (without extension) [$Pwd]: "
        read tar
        [ "$tar"x = x ] && tar="$Pwd"
        cd .. && \
        tar cf - "$Pwd" | gzip -f9 > "$tar.tar.gz" && \
        echo "../$tar.tar.gz created."
7za no redirect.

Code: Select all

5       Compress the current subdirectory (tar.7z)
        Pwd=`basename %d /`
        echo -n "Name of the compressed file (without extension) [$Pwd]: "
        read tar
        [ "$tar"x = x ] && tar="$Pwd"
        cd .. && \
        tar cf - "$Pwd" | 7za a -si "$tar.tar.7z" && \
        echo "../$tar.tar.7z created."
Both are mc original code without parallel compression / threadability.
Cheers!
Yours Rava

Post Reply