Simple Script to Merge Modules

Post tutorials, HOWTO's and other useful resources here.
Bogomips
Full of knowledge
Full of knowledge
Posts: 2564
Joined: 25 Jun 2014, 15:21
Distribution: 3.2.2 Cinnamon & KDE5
Location: London

Simple Script to Merge Modules

Post#1 by Bogomips » 01 Aug 2015, 20:47

Making available this script which will merge modules in a folder. The script suits a situation where one is strapped for RAM. Script will take all the module .xzm files that are to be found in a directory and merge them into a single module file. Modules will have to be decompressed in a work directory, which by default is /tmp. However, depending on the size of the modules, there might not be enough available ram to accomodate them, in which case it would be prudent to furnish another directory on a real (as opposed to live) file system with sufficient spare space.

Quite a few script writers seem to always make the assumption that /tmp is a reservoir of boundless free memory, and do not realise that on this type of system, what goes into temp comes out of home, which could result in everything grinding to a standstill. :twisted:

So enjoy mrgmod (or anything else one might like to call it)!

Code: Select all

#!/bin/bash
# 2015-07-26 00:31:07
mgm()
{
#	Usage:  mrgmod  [-nv]   [./]    [/Path/to/]<merged modules>.xzm  [/tmp]
# mrgmod  [Non-verbose]   [<modules directory>]   [/Path/to/]<merged module>.xzm  [<An Existing Directory for Temporary Files>]
# md: <modules directory>	mf: <merged modules file>.xzm	mt: <Temporary Files Directory>
[[ $(whoami) != root ]] && { echo Please run as root; exit 9; }
# Non-verbose Option?
vb=1; ve=0; op=$1;
[[ $1 ]] && ! [[ ${1%%-*} ]] && { shift; vb=0; [[ $op != -nv ]] && ve=1; }
((ve)) && echo "Incorrect option '$op'!";
for o in $*; do ! [[ ${o%%-*} ]] && ve=1 && echo "Error: Option '$o' Out of Place!"; done
np=$#;	# of parameters
if [[ np -eq 0 || np -gt 3 || ve -eq 1 ]]; then [[ np -gt 3 ]] && echo Error: Too many arguments!
  na=${0##*/}; na=${na%.sh}
  printf "Usage:\t%s\t[-nv]\t[./]\t[/Path/to/]<merged module>.xzm\t[/tmp]\n\n" $na
  printf "%s\t[Non-verbose]\t[<modules directory>]\t[/Path/to/]<merged module>.xzm\t[<An Existing Directory for Temporary Files>]\n" $na
  exit $np
fi
# 
pa=($1 $2 $3); let m=np-1; md=""; mt=/tmp;	# Default modules dir and temp files dir 
# Work out arguments. Brginning from last non-null parameter value.
if [[ -d ${pa[m]} ]]; then	[[ np -eq 1 ]] && { echo Error: No File for Merged Module!; exit 1; }
  mf=${pa[m-1]}; [[ np -eq 3 ]] && md=$1;
else	[[ np -eq 3 ]] && { if ! [[ $3 =~ .+\.xzm$ ]]; then echo "Error: '$3' Non-existent Temporary Files Directory!"; else echo "Error: Superfluous Modules Directory '$2'!"; fi; exit 3; }
  mf=${pa[m]}; [[ np -eq 2 ]] && md=$1;
fi
! [[ $mf =~ .+\.xzm$ ]] && { echo "Error: Module File '$mf' hasn't xzm extension!"; exit 2; }
[[ ! -d ${md:-.} ]] && { echo "Error: Modules Directory '$md' Non-existent!"; exit 3; }
! [[ -w $(dirname $mf) ]] && { echo "Permission: Cannot create Merged Modules File '$mf'!"; exit 2; };
# Create fake root directory
mfr=$mt/mfr$$; mkdir $mfr; (($?)) && { echo "Error: Cannot create Temporary Files in '$mt' Directory!"; exit 3; } 
mm()
{
  [[ $# -lt 2 ]] && { echo "Error: Merging Modules of ${md:-Current} Directory. Less than 2 Modules for Merging!"; return 1; }
  echo $'\n'"Processing Modules of ${md:-Current} Directory."; ((vb)) && echo;
  for a in $*;
  do 
    xzm2dir $a $mfr > /dev/null && { ((vb)) && echo "Processed Module: ${a##*/}"; continue; }
    echo "Error: Processing Module '$a'!"; return 1; 
  done
  echo $'\n'"->  Merged Modules File '$mf'"$'\n';
  dir2xzm $mfr $mf && echo $'\n'"Merged Modules File '$mf' Created." && return 0;
  echo Error: Merging Modules!; return 1;
}
  mm ${md:+$md/}*.xzm; (($?)) && [[ -f $mf ]] && rm $mf; 
  rm -fr $mfr
}
mgm $*
Edited Sun Aug 2 18:17:48 UTC 2015 to ensure smoother run by switching to root.

Invocation Example:

Code: Select all

root@porteus:/home/guest# sh p10/Por/mrgmod.sh /tmp/usm/openjre-7u79_b14 openjre-7u79_b14.xzm /mnt/sda9/Junk    

Processing Modules of /tmp/usm/openjre-7u79_b14 Directory.

Processed Module: aaa_elflibs-14.1-i486-3.xzm
Processed Module: openjre-7u79_b14-i486-2sl.xzm
Processed Module: openssl-solibs-1.0.1e-i486-1.xzm
Processed Module: rhino-1_7R4-i486-3sl.xzm

->  Merged Modules File 'openjre-7u79_b14.xzm'

Parallel mksquashfs: Using 1 processor
Creating 4.0 filesystem on openjre-7u79_b14.xzm, block size 262144.
[=========================================================================|] 1219/1219 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
        compressed data, compressed metadata, compressed fragments, compressed xattrs
        duplicates are removed
Filesystem size 48251.71 Kbytes (47.12 Mbytes)
        53.70% of uncompressed filesystem size (89853.29 Kbytes)
Inode table size 9734 bytes (9.51 Kbytes)
        25.73% of uncompressed inode table size (37832 bytes)
Directory table size 11062 bytes (10.80 Kbytes)
        46.50% of uncompressed directory table size (23790 bytes)
Number of duplicate files found 28
Number of inodes 1120
Number of files 945
Number of fragments 66
Number of symbolic links  79
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 96
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)

Merged Modules File 'openjre-7u79_b14.xzm' Created.
______________________________________________________________________________
______________________________________________________________________________
root@porteus:/home/guest# sh p10/Por/mrgmod.sh -nv /tmp/usm/openjre-7u79_b14 openjre-7u79_b14.xzm /mnt/sda9/Junk

Processing Modules of /tmp/usm/openjre-7u79_b14 Directory.

->  Merged Modules File 'openjre-7u79_b14.xzm'

Parallel mksquashfs: Using 1 processor
Creating 4.0 filesystem on openjre-7u79_b14.xzm, block size 262144.
[=========================================================================|] 1219/1219 100%

Exportable Squashfs 4.0 filesystem, xz compressed, data block size 262144
        compressed data, compressed metadata, compressed fragments, compressed xattrs
        duplicates are removed
Filesystem size 48251.72 Kbytes (47.12 Mbytes)
        53.70% of uncompressed filesystem size (89853.29 Kbytes)
Inode table size 9738 bytes (9.51 Kbytes)
        25.74% of uncompressed inode table size (37832 bytes)
Directory table size 11062 bytes (10.80 Kbytes)
        46.50% of uncompressed directory table size (23790 bytes)
Number of duplicate files found 28
Number of inodes 1120
Number of files 945
Number of fragments 66
Number of symbolic links  79
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 96
Number of ids (unique uids + gids) 1
Number of uids 1
        root (0)
Number of gids 1
        root (0)

Merged Modules File 'openjre-7u79_b14.xzm' Created.
Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB

User avatar
Ed_P
Contributor
Contributor
Posts: 8341
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 5.01 ISO
Location: Western NY, USA

Re: Simple Script to Merge Modules

Post#2 by Ed_P » 02 Aug 2015, 00:46

hmmm This script, and posting, look a lot like one I saw, and saved, yesterday.
Ed

Post Reply