xzmmerge without new compressing?

New features which should be implemented in Porteus; suggestions are welcome. All questions or problems with testing releases (alpha, beta, or rc) should go in their relevant thread here, rather than the Bug Reports section.
Post Reply
User avatar
Rava
Contributor
Contributor
Posts: 1319
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 3.1.0 x86-64 XFCe
Location: Germany

xzmmerge without new compressing?

Post#1 by Rava » 27 Sep 2016, 17:05

Would it be technically possible to create an xzmmerge (add several xzm's into one that contains all) without the need to compress them all, since they all should have a similar file structure already?

So, when I have 3 or 8 modules, some several MB in size, some only kB for single smaller/dependencies, to be able to create the merged mosule in under 5 seconds even when the module is like 80MB in size?
Cheers!
Yours Rava

tome
Contributor
Contributor
Posts: 562
Joined: 26 Jun 2013, 14:03
Distribution: x64 Openbox
Location: Poland
Contact:

Re: xzmmerge without new compressing?

Post#2 by tome » 27 Sep 2016, 19:11

Find line in script that have mksquashfs .... -comp xz and remove "-comp xz"
See also available compressions - type mksquashfs in terminal.

User avatar
wread
Module Guard
Module Guard
Posts: 1080
Joined: 09 Jan 2011, 18:48
Distribution: Porteus v3.2.5-kde5-64 bits
Location: Santo Domingo
Contact:

Re: xzmmerge without new compressing?

Post#3 by wread » 27 Sep 2016, 19:31

I would say it won't work, but try it: "cortando cojones se aprende a capar" (spanisch Sprichtwort)!

Cheers!
Porteus is proud of the FASTEST KDE ever made.....(take akonadi, nepomuk and soprano out and you will have a decent OS).
The Porteus Community never sleeps!

User avatar
Rava
Contributor
Contributor
Posts: 1319
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 3.1.0 x86-64 XFCe
Location: Germany

Re: xzmmerge without new compressing?

Post#4 by Rava » 27 Sep 2016, 19:36

^
So far there is no such script as xzmmerge.

I looked into dir2xzm (that's the only one I have currently that created xzm's) and sure there is no "-comp xz" in there....

I know there is a script for merging modules doing it the xzm2dir xzm2dir and then dir2xzm approach but dunno where to find right now...

Code: Select all

#root@porteus:/# ls /opt/porteus-scripts/*xzm*
/opt/porteus-scripts/deb2xzm  /opt/porteus-scripts/tgz2xzm
/opt/porteus-scripts/dir2xzm  /opt/porteus-scripts/txz2xzm
/opt/porteus-scripts/lsxzm    /opt/porteus-scripts/xzm2dir
/opt/porteus-scripts/rpm2xzm
root@porteus:/# grep -- "-comp xz" /opt/porteus-scripts/*xzm*
root@porteus:/# 
wread wrote:I would say it won't work, but try it: "cortando cojones se aprende a capar" (spanisch Sprichtwort)!
Google translate gives me "cutting fuck it learns to castrate"
Seems a bit off when translated, but it sounds insulting...
Cheers!
Yours Rava

tome
Contributor
Contributor
Posts: 562
Joined: 26 Jun 2013, 14:03
Distribution: x64 Openbox
Location: Poland
Contact:

Re: xzmmerge without new compressing?

Post#5 by tome » 27 Sep 2016, 20:33

Sorry, my mistake, for default compression "-comp xz" is not required. Try edit

Code: Select all

squashfs "$1" "$2" -b 256K -Xbcj x86 -noappend $keep $3 $4 $5 $6 $7 $8 $9
in dir2xzm.

Available for this purpose no-compressions parameters are:
-noInodeCompression alternative name for -noI
-noDataCompression alternative name for -noD
-noFragmentCompression alternative name for -noF
-noXattrCompression alternative name for -noX

User avatar
Rava
Contributor
Contributor
Posts: 1319
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 3.1.0 x86-64 XFCe
Location: Germany

Re: xzmmerge without new compressing?

Post#6 by Rava » 27 Sep 2016, 21:52

^
I think you misunderstood my post. I want the new module to be compressed, I just ask if it is possible to combine already compressed modules into a new one without the need of decompressing and compressing.

I presume it is not possible, but who knows. It can be mounted as it is and it not needs to be decompressed again when a new area of the module is accessed that was not read prior, so I thought maybe...
Cheers!
Yours Rava

User avatar
Ed_P
Contributor
Contributor
Posts: 3205
Joined: 06 Feb 2013, 22:12
Distribution: Cinnamon 3.2.2 64-bit ISO
Location: Western NY, USA

Re: xzmmerge without new compressing?

Post#7 by Ed_P » 27 Sep 2016, 23:58

Rava wrote:is possible to combine already compressed modules into a new one without the need of decompressing and compressing.
A possibility to try that would cut out the decompressing, copy the xzm files to a directory then dir2xzm it to a new module. Try it with a couple of your smaller modules to see if the idea works. :)
Ed

User avatar
brokenman
Site Admin
Site Admin
Posts: 5503
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v3.2rcX all desktops
Location: Brazil
Contact:

Re: xzmmerge without new compressing?

Post#8 by brokenman » 28 Sep 2016, 00:02

I believe it is possible since you can mount two modules as an overlay (on a different branch). There is already a function to append an archive but I don't think it allows for overwriting existing files, only adding new ones. I will see what can be done.

I know that you will always need to re-compress or create the new module, but the initial decompress can just be a mount.

EDIT:
So in essence what you want can't be done since a module is a read only package like an ISO. It must be unpacked (or mounted on overlay) and the new read only archive must be created from the overlayed mounts. The time will be taken to create the new module. So no, not really possible.

https://forum.porteus.org/viewtopic.php ... nch#p26626
How do i become super user?
Wear your underpants on the outside and put on a cape.

User avatar
Rava
Contributor
Contributor
Posts: 1319
Joined: 11 Jan 2011, 02:46
Distribution: Porteus 3.1.0 x86-64 XFCe
Location: Germany

Re: xzmmerge without new compressing?

Post#9 by Rava » 28 Sep 2016, 00:59

brokenman wrote:So in essence what you want can't be done since a module is a read only package like an ISO. It must be unpacked (or mounted on overlay) and the new read only archive must be created from the overlayed mounts. The time will be taken to create the new module. So no, not really possible.
So, the only part that could be done is skipping the decompress part by just mounting the modules, and also the skipping the creation of a new folder, since the script could theoretically be pointed to the mounted overlays fro input and just has to compress and create the new module.
But that is quite the more complex approach than what we do now, and since complexity also gives potential for bugs and errors, we better stay with the way we already do it.
Cheers!
Yours Rava

Post Reply