mergexzm - merge folder of modules and slack packages

[freestyler]

This script merges a folder of modules and slackware packages into one module.
usage:
follow the prompts with:

Code: Select all

mergexzm

or

Code: Select all

mergexzm /path/to/folder/of/modules modulename 

example:

Code: Select all

mergexzm /home/guest/xfcemodules xfce

xfce.xzm will be created in /tmp/mergexzm

download: mergexzm-1.0-no-arch_fs.xzm

[freestyler]

Ive updated the script to include a gui. Here are some screenshots...

If you run 'mergexzm' it will ask for a folder or modules and a name.


It can also be used as 'mergexzm /path/to/folder/of/modules modulename'


The gui is run using 'mergeusm -g'


You can choose modules from various locations and merge them.It also merges tgz and txz slackware packages


The gui can also be used to merge a folder of modules and slackware packages.


Download: mergexzm-1.1-no-arch_fs.xzm

Code: Select all

#!/bin/bash

# mergexzm merges a folder of xzm modules and slackware packages into one module
# for graphical user interface use: 'mergexzm -g'
# usage: mergexzm /path/to/folder/of/modules modulename
# example: mergexzm /home/guest/xfcemodules xfce
# xfce.xzm will be created in /tmp/mergexzm

# version 1.1

# script by freestyler <mail@porteus-apps.org>


# make sure user has root access

if [ $UID -ne 0 ]; then
	echo; echo "you must have root access to merge modules."; echo; exit; 
fi

helptext(){ 
	echo ""
	echo "==============================================================================="
	echo "!                                  mergexzm                                   !"
	echo "==============================================================================="
	echo ""
	echo "                                  version 1.1"
	echo ""
	echo "                  mergexzm merges a folder of xzm modules and" 
	echo "                       slackware packages into one module"
	echo ""
	echo "	        for graphical user interface use: 'mergexzm -g'"
	echo ""
	echo "             usage: 'mergexzm /path/to/folder/of/modules modulename'"
	echo ""
	echo "                example: mergexzm /home/guest/xfcemodules xfce"
	echo "                  xfce.xzm will be created in /tmp/mergexzm"
	echo ""
	echo "==============================================================================="
	echo "!                 script by freestyler <mail@porteus-apps.org>                !"
	echo "==============================================================================="
	echo ""
}

start(){
	echo ""
	echo "==============================================================================="
	echo "!                                  mergexzm                                   !"
	echo "==============================================================================="
	echo ""
	echo "                  mergexzm merges a folder of xzm modules and" 
	echo "                       slackware packages into one module"
	echo ""
	echo "                           for help use: 'mergexzm -h'"
	echo ""
	echo "	        for graphical user interface use: 'mergexzm -g'"
	echo ""
}

processfiles(){
	cd /tmp/mergexzm/merged
	echo; echo "Processing files from "$directory""
	echo ""
	for file in *; do
		case `echo $file |awk -F . '{if (NF>1) {print $NF}}'` in
		txz*)  # Convert and merge txz files
			echo "$file is being converted"
			txz2xzm $file &>/dev/null
 	      ;;
		tgz*)  # Convert and merge txz files
			echo "$file is being converted"
		        tgz2xzm $file &>/dev/null
              ;;		
		xzm*)  # xzm module		
 	      ;;
		*) # Anything else
			echo "$file is not a valid module or slackware package."
			continue
              ;;
	esac
	done
}

processxzm(){
	echo ""
	echo "Modules to be merged:"
	cd /tmp/mergexzm/merged
	for file in *; do
	case `echo $file |awk -F . '{if (NF>1) {print $NF}}'` in
		xzm*)  # xzm module
			echo "$file"
			xzm2dir $file /tmp/mergexzm/tmp/ &>/dev/null
              		;;
	esac
	done
}

merge(){
	modulename="$mergeNAME"
	echo "$mergeNAME"
	processfiles
	processxzm
	createmodule
	cleanup
}

dothatshit(){
	cleanup
	cat /tmp/mergexzm/list.txt |
	while read a; do
	cp $a /tmp/mergexzm/merged
	done
	gtkdialog --program=TERMINAL2
}

clearlist(){
	echo > /tmp/mergexzm/list.txt
	rm -rf /tmp/mergexzm/list.txt
	cat > /tmp/mergexzm/list.txt &
}

createmodule(){
	cd /tmp/mergexzm
	echo ""
	echo "==============================================================================="
	echo "!                               Creating module                               !"
	echo "==============================================================================="
	echo ""
	dir2xzm /tmp/mergexzm/tmp "$modulename".xzm
	echo ""
	echo "==============================================================================="
	echo "!                                  Complete!                                  !"
	echo "==============================================================================="
	echo ""
	echo ""$modulename".xzm has been created in /tmp/mergexzm"
	echo ""
}

cleanup(){
	if [[ ! -d "/tmp/mergexzm" ]]; then 
		mkdir /tmp/mergexzm
	fi

	if [[ -d "/tmp/mergexzm/tmp" ]]; then
		rm -rf /tmp/mergexzm/tmp
		mkdir /tmp/mergexzm/tmp
		else
		mkdir /tmp/mergexzm/tmp
	fi

	if [[ -d "/tmp/mergexzm/merged" ]]; then
		rm -rf /tmp/mergexzm/merged
		mkdir /tmp/mergexzm/merged
	fi
}

export MERGEGUI='
<window window_position="1" title="mergexzm" width_request="380" >
  <vbox>   
    <button space-expand="false" space-fill="false">
      <label>Select Modules To Merge</label>
      <action>gtkdialog --program=SELECTMODULES</action></button>
    <button space-expand="false" space-fill="false">
      <label>Select A Folder Of Modules to Merge</label>
      <action>gtkdialog --program=SELECTFOLDER</action></button>
    <button space-expand="false" 
      space-fill="false">
      <label>Exit</label></button>
  </vbox>
</window>
'

export SELECTMODULES='
<window window_position="1" title="Select modules to merge" width_request="540" height_request="380">
  <vbox>
    <tree auto-refresh="true">
      <label>Modules to be merged</label>
      <height>100</height>
      <width>200</width>
      <variable>mergeLIST</variable>
      <input file>/tmp/mergexzm/list.txt</input>
    </tree>
    <hbox>
      <button><label>Clear list</label>
        <action>clearlist</action></button>
      <entry fs-action="file"
	fs-title="Select a file"
	fs-filters="*.'xzm'|*.'tgz'|*.'txz'"
	fs-folder="'$HOME'"
	editable="true">
        <variable>selectMODULE</variable></entry>
      <button><label>Browse</label>
	<action function="fileselect">selectMODULE</action></button>
      <button><label>Add</label>
	<action>echo "$selectMODULE" >> /tmp/mergexzm/list.txt</action></button>
    </hbox>
    <hbox><text><label>Module Name:</label></text>
      <entry editable="true">
        <variable>mergeNAME</variable></entry>
      <button space-expand="false" space-fill="false">
        <label>Merge</label>
        <action>dothatshit</action></button>
      <button space-expand="false" space-fill="false">
        <label>Cancel</label></button>
    </hbox>
  </vbox>
</window>
'

export SELECTFOLDER='
<window window_position="1" title="Select folder to merge" width_request="380" >
  <vbox>   
    <text><label>This will merge all xzm modules and slackware packages contained in selected folder. Please choose a name for the module</label></text>
      <hbox>	
	<text space-expand="false" space-fill="false">
	  <label>Name:</label></text>
	<entry editable="true">
          <variable>mergeNAME</variable></entry>
      </hbox>
      <hbox>	
        <entry fs-action="folder"
	  space-expand="true"
	  space-fill="true"
	  fs-title="Select a folder"
	  fs-folder="'/home/guest/'"
	  editable="true">
          <default>/path/to/folder/containing/modules</default>
          <variable>mergeDIR</variable></entry>
        <button space-expand="false" space-fill="false">
	  <label>Browse</label>
	  <action function="fileselect">mergeDIR</action></button>
      </hbox>
    <button space-expand="false" space-fill="false">
      <label>Merge</label>
      <variable>butselFile</variable>
      <action>gtkdialog --program=TERMINAL</action></button>
    <button space-expand="false" space-fill="false">
      <label>Cancel</label></button>
  </vbox>
</window>
'

export TERMINAL='
<window window_position="1" title="Merging Modules" width_request="680" height_request="400" icon-name="media-playback-start">
  <vbox>
    <hbox>
      <terminal tag_attr="value" vscrollbar-policy="0">  
        <input>echo mergexzm "$mergeDIR" "$mergeNAME"</input>
      </terminal>
    </hbox>
  </vbox>
</window>
'

export TERMINAL2='
<window window_position="1" title="Merging Modules" width_request="680" height_request="400" icon-name="media-playback-start">
  <vbox>
    <hbox>
      <terminal tag_attr="value" vscrollbar-policy="0">  
        <input>echo merge</input>
      </terminal>
    </hbox>
  </vbox>
</window>
'

export -f merge
export -f clearlist
export -f dothatshit
export -f cleanup
export -f processfiles
export -f processxzm
export -f createmodule
export -f start
export -f helptext

# set help screen and start script

[[ $@ ]] || { start; }

while getopts "hg" opt; do
  case $opt in
    h) helptext >&2
      exit 1
    ;;
    g) gtkdialog --program=MERGEGUI >&2
      exit 1
    ;;
    \?) echo "for help use: mergexzm -h" >&2
      exit 1
    ;;
    :) echo "Option -$OPTARG requires an argument." >&2
      exit 1
    ;;
    *) start
      exit 1	    
    ;;
  esac
done

# check if user has entered variables

if [ -z "$1" ]; then
	echo "Enter location of folder containing modules you wish to merge:"
	read directory
	echo "Modules in "$directory" will be merged"
 	else
	export directory="$1"
fi

if [ -z "$2" ]; then
	echo "Please choose a name for your module:"
	read modulename
	echo "Module will be named "$modulename".xzm"
 	else
	export modulename="$2"
fi

# clean temp files and create needed temporary folders

cleanup

# copy contents of directory to temporary loctation

cp $(find $directory -type f | grep -iE '\.(xzm|tgz|txz)$') /tmp/mergexzm/merged

# convert slackware packages to modules and then extract modules to temp directory

processfiles
processxzm

# create xzm module

createmodule
cleanup

[brokenman]

Nice contribution. Will come in handy.

[freestyler]

Fixed an initialization problem (was not creating a necessary temp file when using gui)
I updated the download file

[Payoon]

Hi Freestyler,
thanks a lot for this nice little programm. It's very useful. When I use the built-in merge function of porteus the path is to /porteus/modules, where other active modules are and to keep things separate is sometimes tricky. With merge2xzm I can create a folder with the new module name, throw all my converted dependency deb's and txz's into it, open a terminal inside this folder et voila! a new module_alldeps.xzm is born. Accordingly I have merged it into my own myutilites.xzm for everyday use and things work fine.
Payoon

[francois]

This is a nice tool. I will use it for sure.

Thanks.

[francois]

1) It does not work for me:

Code: Select all

root@porteus:~# mergexzm /mnt/sda5/64bit_v31rc1/porteus/optional/gimp-2.8.6 /tmp/gimp-2.8.6.xzm

I get a module but with nothing in it.

2) the gui is not functioning either:

Code: Select all

root@porteus:~# mergeusm -g
bash: mergeusm: command not found

[donald]

^
# for graphical user interface use: 'mergexzm -g'

[port]

A potencial problem I see is order of merging. It seems you merge the modules in the order you read them from disk but this may not be the right idea because you can't control which module hides which else and that's not good.

There's not a right solution for the problem as the process being automated but taking into account date would probably be a good idea and also a kind of sorting based in file names will give the opportunity to user to stablish a desired order of appliance, it would be great if this were available as parameter or config

For reasons like that I think a better solution is to manually unpack the files to dir in the order you wish (maybe after analyzing contents) and tweaking manually dir contents to be sure it's all ok and then build the module from dir. Maybe another good options for the tools is automate the process of unpacking without building the module after user validation, maybe dividing the script in several stateful stages in the *ctl way with commands to init each stage (i.e. git, mercurial, sysctl...)

[rych]

porteux, thanks for adding the mergexzm to PorteuX 1.4. From the performance (of SquashFS, aufs mounts, ...) point of view, is it batter to have 20 small .xzms or one large .XZM module?

[Vic]

Good question Rych. I have quite a few (50) small modules around 4KiB in size and was wondering the same thing. Having individual modules makes it alot easier to organize and modify a single parameter.

Vic

[roadie]

I merged my collection of modules into one large module. The upshot was, from the standpoint of performance, the difference wasn't enough to be worth the effort. It also becomes problematic when the files contained in any certain module cause problems. A while back, I had problems with Filezilla, it wouldn't start if the keypassx module was activated. With merged modules, it's pretty hard to track down what's causing problems.

The performance difference amounted to no more than 2 seconds judging by boot times, which themselves vary. Even with individual modules, it cost 2 hours to find that keypassx was the problem......I can't imagine the headaches with a merged module.

[Vic]

I just counted the modules that load everytime I boot Porteux: 119

Many are config files for Plasma and the programs I always use. Also application and service menu .desktop files.

I discovered that the number prefix not only works for .xzm files but also folders, so I have a hierarchy of numbered folders too.

With the cheatcodes I can load any folders to find a problem or just try something new out.

As Roadie posted, troubleshooting is much easier with individual modules.

Vic

[porteux]

porteux, thanks for adding the mergexzm to PorteuX 1.4. From the performance (of SquashFS, aufs mounts, ...) point of view, is it batter to have 20 small .xzms or one large .XZM module?

Considering the modules loading time, it's much faster to have a big one than many tiny ones.

The size of a module shouldn't impact its loading time, only the amount of files it contains inside. Still, it would be faster to load a merged one with everything than many little ones.

Anyway, we're probably talking about a few seconds difference here, depending on how many modules we're loading. Most likely not a big deal.

But, as Vic and roadie said, merging modules can be problematic for other reasons.