converting .txz files into .xzm for a whole folder

For discussions about programming and projects not necessarily associated with Porteus.

Re: converting .txz files into .xzm for a whole folder

Postby Bogomips » 31 Jul 2015, 17:00

francois wrote:So some kind of function to merge xzms.
Would be surprised if francois was unaware of the existing merge script. Anyway, regarding this as a programming exercise, keeping one's Bash in trim, have tried to incorporate the two simple statements into a function, and have come up with a simple script which should do the trick. Hope all bases covered. :Search:

Being somewhat chary of using others scripts, it will also serve as a script in reserve for me. The script writers seem to always make the assumption that /tmp is a reservoir of unlimited free memory, and do not realise that on this system, what goes into temp comes out of my home, which could result in everything grinding to a standstill. :twisted: Due to this assumption, neko's script did not work, but he is changing it now to allow for an option to take this into account.

So if you wish to have yet another script, enjoy mrgmod!
Script Updated: https://forum.porteus.org/viewtopic.php?f=39&t=4896
Code: Select all
#!/bin/bash
# 2015-07-26 00:31:07
mgm()
{
#   Usage:  mrgmod  [-nv]   [./]    [/Path/to/]<merged modules>.xzm  [/tmp]
# mrgmod  [Non-verbose]   [<modules directory>]   [/Path/to/]<merged module>.xzm  [<An Existing Directory for Temporary Files>]
# md: <modules directory>   mf: <merged modules file>.xzm   mt: <Temporary Files Directory>
# Non-verbose Option?
vb=1; ve=0; op=$1;
[[ $1 ]] && ! [[ ${1%%-*} ]] && { shift; vb=0; [[ $op != -nv ]] && ve=1; }
((ve)) && echo "Incorrect option '$op'!";
for o in $*; do ! [[ ${o%%-*} ]] && ve=1 && echo "Error: Option '$o' Out of Place!"; done
np=$#;   # of parameters
if [[ np -eq 0 || np -gt 3 || ve -eq 1 ]]; then [[ np -gt 3 ]] && echo Error: Too many arguments!
  na=${0##*/}; na=${na%.sh}
  printf "Usage:\t%s\t[-nv]\t[./]\t[/Path/to/]<merged module>.xzm\t[/tmp]\n\n" $na
  printf "%s\t[Non-verbose]\t[<modules directory>]\t[/Path/to/]<merged module>.xzm\t[<An Existing Directory for Temporary Files>]\n" $na
  exit $np
fi
#
pa=($1 $2 $3); let m=np-1; md=""; mt=/tmp;   # Default modules dir and temp files dir
# Work out arguments. Brginning from last non-null parameter value.
if [[ -d ${pa[m]} ]]; then   [[ np -eq 1 ]] && { echo Error: No File for Merged Module!; exit 1; }
  mf=${pa[m-1]}; [[ np -eq 3 ]] && md=$1;
else   [[ np -eq 3 ]] && { if ! [[ $3 =~ .+\.xzm$ ]]; then echo "Error: '$3' Non-existent Temporary Files Directory!"; else echo "Error: Superfluous Modules Directory '$2'!"; fi; exit 3; }
  mf=${pa[m]}; [[ np -eq 2 ]] && md=$1;
fi
! [[ $mf =~ .+\.xzm$ ]] && { echo "Error: Module File '$mf' hasn't xzm extension!"; exit 2; }
[[ ! -d ${md:-.} ]] && { echo "Error: Modules Directory '$md' Non-existent!"; exit 3; }
! [[ -w $(dirname $mf) ]] && { echo "Permission: Cannot create Merged Modules File '$mf'!"; exit 2; };
# Create fake root directory
mfr=$mt/mfr$$; mkdir $mfr; (($?)) && { echo "Error: Cannot create Temporary Files in '$mt' Directory!"; exit 3; }
mm()
{
  [[ $# -lt 2 ]] && { echo "Error: Merging Modules of ${md:-Current} Directory. Less than 2 Modules for Merging!"; return 1; }
  echo $'\n'"Processing Modules of ${md:-Current} Directory."; ((vb)) && echo;
  for a in $*;
  do
    xzm2dir $a $mfr > /dev/null && { ((vb)) && echo "Processed Module: ${a##*/}"; continue; }
    echo "Error: Processing Module '$a'!"; return 1;
  done
  echo $'\n'"->  Merged Modules File '$mf'"$'\n';
  dir2xzm $mfr $mf && echo $'\n'"Merged Modules File '$mf' Created." && return 0;
  echo Error: Merging Modules!; return 1;
}
  mm ${md:+$md/}*.xzm; (($?)) && [[ -f $mf ]] && rm $mf;
  rm -fr $mfr
}
mgm $*


Invocation Example:
Code: Select all
root@porteus:/home/guest# sh p10/Por/mrgmod.sh /tmp/usm/openjre-7u79_b14 openjre-7u79_b14.xzm /mnt/sda9/Junk   

Processing Modules of /tmp/usm/openjre-7u79_b14 Directory.

Processed Module: aaa_elflibs-14.1-i486-3.xzm
Processed Module: openjre-7u79_b14-i486-2sl.xzm
Processed Module: openssl-solibs-1.0.1e-i486-1.xzm
Processed Module: rhino-1_7R4-i486-3sl.xzm

->  Merged Modules File 'openjre-7u79_b14.xzm'

Parallel mksquashfs: Using 1 processor
Creating 4.0 filesystem on openjre-7u79_b14.xzm, block size 262144.
[=========================================================================|] 1219/1219 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
        compressed data, compressed metadata, compressed fragments, compressed xattrs
        duplicates are removed
Filesystem size 48251.71 Kbytes (47.12 Mbytes)
        53.70% of uncompressed filesystem size (89853.29 Kbytes)
Inode table size 9734 bytes (9.51 Kbytes)
        25.73% of uncompressed inode table size (37832 bytes)
Directory table size 11062 bytes (10.80 Kbytes)
        46.50% of uncompressed directory table size (23790 bytes)
Number of duplicate files found 28
Number of inodes 1120
Number of files 945
Number of fragments 66
Number of symbolic links  79
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 96
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)

Merged Modules File 'openjre-7u79_b14.xzm' Created.
______________________________________________________________________________
______________________________________________________________________________
root@porteus:/home/guest# sh p10/Por/mrgmod.sh -nv /tmp/usm/openjre-7u79_b14 openjre-7u79_b14.xzm /mnt/sda9/Junk

Processing Modules of /tmp/usm/openjre-7u79_b14 Directory.

->  Merged Modules File 'openjre-7u79_b14.xzm'

Parallel mksquashfs: Using 1 processor
Creating 4.0 filesystem on openjre-7u79_b14.xzm, block size 262144.
[=========================================================================|] 1219/1219 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
        compressed data, compressed metadata, compressed fragments, compressed xattrs
        duplicates are removed
Filesystem size 48251.72 Kbytes (47.12 Mbytes)
        53.70% of uncompressed filesystem size (89853.29 Kbytes)
Inode table size 9738 bytes (9.51 Kbytes)
        25.74% of uncompressed inode table size (37832 bytes)
Directory table size 11062 bytes (10.80 Kbytes)
        46.50% of uncompressed directory table size (23790 bytes)
Number of duplicate files found 28
Number of inodes 1120
Number of files 945
Number of fragments 66
Number of symbolic links  79
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 96
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)

Merged Modules File 'openjre-7u79_b14.xzm' Created.
Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB
Bogomips
Full of knowledge
Full of knowledge
 
Posts: 2243
Joined: 25 Jun 2014, 16:21
Location: London
Distribution: 3.2.2 Cinnamon & KDE5

Re: converting .txz files into .xzm for a whole folder

Postby francois » 31 Jul 2015, 22:33

Thanks all. This is comforting to have so many hands on that problem of mine. :)
Voltaire: Le mieux est l'ennemi du bien.
User avatar
francois
Contributor
Contributor
 
Posts: 4681
Joined: 28 Dec 2010, 15:25
Location: C'est l'autome, en forêt les arbres d'ici nous en mettent plein la vue: les rouges, jaunes ...
Distribution: kde xfce porteus manjaro kubun

Re: converting .txz files into .xzm for a whole folder

Postby francois » 10 Jan 2016, 17:05

Here is the answer to converting a whole folder of txz to a single module:
use usm tools, there is a function to do that exactly.

Also usefull is freestyler mergexzm, that converts a bunch of modules into a single merged module:
https://forum.porteus.org/viewtopic.php?f=75&t=3532
Voltaire: Le mieux est l'ennemi du bien.
User avatar
francois
Contributor
Contributor
 
Posts: 4681
Joined: 28 Dec 2010, 15:25
Location: C'est l'autome, en forêt les arbres d'ici nous en mettent plein la vue: les rouges, jaunes ...
Distribution: kde xfce porteus manjaro kubun

Previous

Return to Programming



Who is online

Users browsing this forum: No registered users and 1 guest