Page 1 of 1
mergexzm - merge folder of modules and slack packages
Posted: 01 Jun 2014, 01:03
by freestyler
This script merges a folder of modules and slackware packages into one module.
usage:
follow the prompts with:
or
Code: Select all
mergexzm /path/to/folder/of/modules modulename
example:
Code: Select all
mergexzm /home/guest/xfcemodules xfce
xfce.xzm will be created in /tmp/mergexzm
download: mergexzm-1.0-no-arch_fs.xzm
Re: mergexzm - merge folder of modules and slack packages
Posted: 02 Jun 2014, 00:12
by freestyler
Ive updated the script to include a gui. Here are some screenshots...
If you run 'mergexzm' it will ask for a folder or modules and a name.
It can also be used as 'mergexzm /path/to/folder/of/modules modulename'
The gui is run using 'mergeusm -g'
You can choose modules from various locations and merge them.It also merges tgz and txz slackware packages
The gui can also be used to merge a folder of modules and slackware packages.
Download: mergexzm-1.1-no-arch_fs.xzm
Code: Select all
#!/bin/bash
# mergexzm merges a folder of xzm modules and slackware packages into one module
# for graphical user interface use: 'mergexzm -g'
# usage: mergexzm /path/to/folder/of/modules modulename
# example: mergexzm /home/guest/xfcemodules xfce
# xfce.xzm will be created in /tmp/mergexzm
# version 1.1
# script by freestyler <[email protected]>
# make sure user has root access
if [ $UID -ne 0 ]; then
echo; echo "you must have root access to merge modules."; echo; exit;
fi
helptext(){
echo ""
echo "==============================================================================="
echo "! mergexzm !"
echo "==============================================================================="
echo ""
echo " version 1.1"
echo ""
echo " mergexzm merges a folder of xzm modules and"
echo " slackware packages into one module"
echo ""
echo " for graphical user interface use: 'mergexzm -g'"
echo ""
echo " usage: 'mergexzm /path/to/folder/of/modules modulename'"
echo ""
echo " example: mergexzm /home/guest/xfcemodules xfce"
echo " xfce.xzm will be created in /tmp/mergexzm"
echo ""
echo "==============================================================================="
echo "! script by freestyler <[email protected]> !"
echo "==============================================================================="
echo ""
}
start(){
echo ""
echo "==============================================================================="
echo "! mergexzm !"
echo "==============================================================================="
echo ""
echo " mergexzm merges a folder of xzm modules and"
echo " slackware packages into one module"
echo ""
echo " for help use: 'mergexzm -h'"
echo ""
echo " for graphical user interface use: 'mergexzm -g'"
echo ""
}
processfiles(){
cd /tmp/mergexzm/merged
echo; echo "Processing files from "$directory""
echo ""
for file in *; do
case `echo $file |awk -F . '{if (NF>1) {print $NF}}'` in
txz*) # Convert and merge txz files
echo "$file is being converted"
txz2xzm $file &>/dev/null
;;
tgz*) # Convert and merge txz files
echo "$file is being converted"
tgz2xzm $file &>/dev/null
;;
xzm*) # xzm module
;;
*) # Anything else
echo "$file is not a valid module or slackware package."
continue
;;
esac
done
}
processxzm(){
echo ""
echo "Modules to be merged:"
cd /tmp/mergexzm/merged
for file in *; do
case `echo $file |awk -F . '{if (NF>1) {print $NF}}'` in
xzm*) # xzm module
echo "$file"
xzm2dir $file /tmp/mergexzm/tmp/ &>/dev/null
;;
esac
done
}
merge(){
modulename="$mergeNAME"
echo "$mergeNAME"
processfiles
processxzm
createmodule
cleanup
}
dothatshit(){
cleanup
cat /tmp/mergexzm/list.txt |
while read a; do
cp $a /tmp/mergexzm/merged
done
gtkdialog --program=TERMINAL2
}
clearlist(){
echo > /tmp/mergexzm/list.txt
rm -rf /tmp/mergexzm/list.txt
cat > /tmp/mergexzm/list.txt &
}
createmodule(){
cd /tmp/mergexzm
echo ""
echo "==============================================================================="
echo "! Creating module !"
echo "==============================================================================="
echo ""
dir2xzm /tmp/mergexzm/tmp "$modulename".xzm
echo ""
echo "==============================================================================="
echo "! Complete! !"
echo "==============================================================================="
echo ""
echo ""$modulename".xzm has been created in /tmp/mergexzm"
echo ""
}
cleanup(){
if [[ ! -d "/tmp/mergexzm" ]]; then
mkdir /tmp/mergexzm
fi
if [[ -d "/tmp/mergexzm/tmp" ]]; then
rm -rf /tmp/mergexzm/tmp
mkdir /tmp/mergexzm/tmp
else
mkdir /tmp/mergexzm/tmp
fi
if [[ -d "/tmp/mergexzm/merged" ]]; then
rm -rf /tmp/mergexzm/merged
mkdir /tmp/mergexzm/merged
fi
}
export MERGEGUI='
<window window_position="1" title="mergexzm" width_request="380" >
<vbox>
<button space-expand="false" space-fill="false">
<label>Select Modules To Merge</label>
<action>gtkdialog --program=SELECTMODULES</action></button>
<button space-expand="false" space-fill="false">
<label>Select A Folder Of Modules to Merge</label>
<action>gtkdialog --program=SELECTFOLDER</action></button>
<button space-expand="false"
space-fill="false">
<label>Exit</label></button>
</vbox>
</window>
'
export SELECTMODULES='
<window window_position="1" title="Select modules to merge" width_request="540" height_request="380">
<vbox>
<tree auto-refresh="true">
<label>Modules to be merged</label>
<height>100</height>
<width>200</width>
<variable>mergeLIST</variable>
<input file>/tmp/mergexzm/list.txt</input>
</tree>
<hbox>
<button><label>Clear list</label>
<action>clearlist</action></button>
<entry fs-action="file"
fs-title="Select a file"
fs-filters="*.'xzm'|*.'tgz'|*.'txz'"
fs-folder="'$HOME'"
editable="true">
<variable>selectMODULE</variable></entry>
<button><label>Browse</label>
<action function="fileselect">selectMODULE</action></button>
<button><label>Add</label>
<action>echo "$selectMODULE" >> /tmp/mergexzm/list.txt</action></button>
</hbox>
<hbox><text><label>Module Name:</label></text>
<entry editable="true">
<variable>mergeNAME</variable></entry>
<button space-expand="false" space-fill="false">
<label>Merge</label>
<action>dothatshit</action></button>
<button space-expand="false" space-fill="false">
<label>Cancel</label></button>
</hbox>
</vbox>
</window>
'
export SELECTFOLDER='
<window window_position="1" title="Select folder to merge" width_request="380" >
<vbox>
<text><label>This will merge all xzm modules and slackware packages contained in selected folder. Please choose a name for the module</label></text>
<hbox>
<text space-expand="false" space-fill="false">
<label>Name:</label></text>
<entry editable="true">
<variable>mergeNAME</variable></entry>
</hbox>
<hbox>
<entry fs-action="folder"
space-expand="true"
space-fill="true"
fs-title="Select a folder"
fs-folder="'/home/guest/'"
editable="true">
<default>/path/to/folder/containing/modules</default>
<variable>mergeDIR</variable></entry>
<button space-expand="false" space-fill="false">
<label>Browse</label>
<action function="fileselect">mergeDIR</action></button>
</hbox>
<button space-expand="false" space-fill="false">
<label>Merge</label>
<variable>butselFile</variable>
<action>gtkdialog --program=TERMINAL</action></button>
<button space-expand="false" space-fill="false">
<label>Cancel</label></button>
</vbox>
</window>
'
export TERMINAL='
<window window_position="1" title="Merging Modules" width_request="680" height_request="400" icon-name="media-playback-start">
<vbox>
<hbox>
<terminal tag_attr="value" vscrollbar-policy="0">
<input>echo mergexzm "$mergeDIR" "$mergeNAME"</input>
</terminal>
</hbox>
</vbox>
</window>
'
export TERMINAL2='
<window window_position="1" title="Merging Modules" width_request="680" height_request="400" icon-name="media-playback-start">
<vbox>
<hbox>
<terminal tag_attr="value" vscrollbar-policy="0">
<input>echo merge</input>
</terminal>
</hbox>
</vbox>
</window>
'
export -f merge
export -f clearlist
export -f dothatshit
export -f cleanup
export -f processfiles
export -f processxzm
export -f createmodule
export -f start
export -f helptext
# set help screen and start script
[[ $@ ]] || { start; }
while getopts "hg" opt; do
case $opt in
h) helptext >&2
exit 1
;;
g) gtkdialog --program=MERGEGUI >&2
exit 1
;;
\?) echo "for help use: mergexzm -h" >&2
exit 1
;;
:) echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
*) start
exit 1
;;
esac
done
# check if user has entered variables
if [ -z "$1" ]; then
echo "Enter location of folder containing modules you wish to merge:"
read directory
echo "Modules in "$directory" will be merged"
else
export directory="$1"
fi
if [ -z "$2" ]; then
echo "Please choose a name for your module:"
read modulename
echo "Module will be named "$modulename".xzm"
else
export modulename="$2"
fi
# clean temp files and create needed temporary folders
cleanup
# copy contents of directory to temporary loctation
cp $(find $directory -type f | grep -iE '\.(xzm|tgz|txz)$') /tmp/mergexzm/merged
# convert slackware packages to modules and then extract modules to temp directory
processfiles
processxzm
# create xzm module
createmodule
cleanup
Re: mergexzm - merge folder of modules and slack packages
Posted: 02 Jun 2014, 02:49
by brokenman
Nice contribution. Will come in handy.
Re: mergexzm - merge folder of modules and slack packages
Posted: 02 Jun 2014, 14:42
by freestyler
Fixed an initialization problem (was not creating a necessary temp file when using gui)
I updated the download file
Re: mergexzm - merge folder of modules and slack packages
Posted: 08 Jun 2014, 12:35
by Payoon
Hi Freestyler,
thanks a lot for this nice little programm. It's very useful. When I use the built-in merge function of porteus the path is to /porteus/modules, where other active modules are and to keep things separate is sometimes tricky. With merge2xzm I can create a folder with the new module name, throw all my converted dependency deb's and txz's into it, open a terminal inside this folder et voila! a new module_alldeps.xzm is born. Accordingly I have merged it into my own myutilites.xzm for everyday use and things work fine.
Payoon
Re: mergexzm - merge folder of modules and slack packages
Posted: 12 Jun 2014, 02:54
by francois
This is a nice tool. I will use it for sure.
Thanks.
Re: mergexzm - merge folder of modules and slack packages
Posted: 10 Jan 2016, 16:18
by francois
1) It does not work for me:
Code: Select all
root@porteus:~# mergexzm /mnt/sda5/64bit_v31rc1/porteus/optional/gimp-2.8.6 /tmp/gimp-2.8.6.xzm
I get a module but with nothing in it.
2) the gui is not functioning either:
Code: Select all
root@porteus:~# mergeusm -g
bash: mergeusm: command not found
Re: mergexzm - merge folder of modules and slack packages
Posted: 10 Jan 2016, 17:41
by donald
^
# for graphical user interface use: 'merge
xzm -g'

Re: mergexzm - merge folder of modules and slack packages
Posted: 12 Mar 2016, 02:26
by port
A potencial problem I see is order of merging. It seems you merge the modules in the order you read them from disk but this may not be the right idea because you can't control which module hides which else and that's not good.
There's not a right solution for the problem as the process being automated but taking into account date would probably be a good idea and also a kind of sorting based in file names will give the opportunity to user to stablish a desired order of appliance, it would be great if this were available as parameter or config
For reasons like that I think a better solution is to manually unpack the files to dir in the order you wish (maybe after analyzing contents) and tweaking manually dir contents to be sure it's all ok and then build the module from dir. Maybe another good options for the tools is automate the process of unpacking without building the module after user validation, maybe dividing the script in several stateful stages in the *ctl way with commands to init each stage (i.e. git, mercurial, sysctl...)
mergexzm - merge folder of modules and slack packages
Posted: 17 Jun 2024, 07:03
by rych
porteux, thanks for adding the mergexzm to PorteuX 1.4. From the performance (of SquashFS, aufs mounts, ...) point of view, is it batter to have 20 small .xzms or one large .XZM module?
mergexzm - merge folder of modules and slack packages
Posted: 17 Jun 2024, 12:58
by Vic
Good question Rych. I have quite a few (50) small modules around 4KiB in size and was wondering the same thing. Having individual modules makes it alot easier to organize and modify a single parameter.
Vic
mergexzm - merge folder of modules and slack packages
Posted: 17 Jun 2024, 18:08
by roadie
I merged my collection of modules into one large module. The upshot was, from the standpoint of performance, the difference wasn't enough to be worth the effort. It also becomes problematic when the files contained in any certain module cause problems. A while back, I had problems with Filezilla, it wouldn't start if the keypassx module was activated. With merged modules, it's pretty hard to track down what's causing problems.
The performance difference amounted to no more than 2 seconds judging by boot times, which themselves vary. Even with individual modules, it cost 2 hours to find that keypassx was the problem......I can't imagine the headaches with a merged module.
mergexzm - merge folder of modules and slack packages
Posted: 17 Jun 2024, 22:56
by Vic
I just counted the modules that load everytime I boot Porteux: 119
Many are config files for Plasma and the programs I always use. Also application and service menu .desktop files.
I discovered that the number prefix not only works for .xzm files but also folders, so I have a hierarchy of numbered folders too.
With the cheatcodes I can load any folders to find a problem or just try something new out.
As Roadie posted, troubleshooting is much easier with individual modules.
Vic
mergexzm - merge folder of modules and slack packages
Posted: 18 Jun 2024, 22:03
by porteux
rych wrote: ↑17 Jun 2024, 07:03
porteux, thanks for adding the
mergexzm to PorteuX 1.4. From the performance (of SquashFS, aufs mounts, ...) point of view, is it batter to have 20 small .xzms or one large .XZM module?
Considering the modules loading time, it's much faster to have a big one than many tiny ones.
The size of a module shouldn't impact its loading time, only the amount of files it contains inside. Still, it would be faster to load a merged one with everything than many little ones.
Anyway, we're probably talking about a few seconds difference here, depending on how many modules we're loading. Most likely not a big deal.
But, as Vic and roadie said, merging modules can be problematic for other reasons.
mergexzm - merge folder of modules and slack packages
Posted: 16 Dec 2024, 03:21
by dreadbird
Yes the idea is that modules should be what you deal in. for a program put all the modules for that in its own folder. /porteus/modules/program1. then as you go you realize you have a common module from another program drop those modules down to /modules and just have 1.
I ran into a very frustrating condition. dependencies are listen on repos and pkgs.org ect but they dont tell you what version of the dependency you need. multiple version of a program use different versions of modules. it becomes very very confusing.
so as you build a new program you experiment with all available versions until you get it to work. then take those files and put it in /program1 this program works.
so at the end of the day combining modules is not good except for local system use cases. I much prefer having modules.